Decoding apparatus and decoding method, and coding apparatus and coding method

ABSTRACT

The present disclosure relates to a decoding apparatus and a decoding method and a coding apparatus and a coding method capable of adjusting a compression rate of a lowest limit. A decoding unit decodes a bit stream coded according to an H.265/HEVC standard having a profile in which a lowest compression rate at the time of coding an image is set for each of a main tier and a high tier in units of Coding Units (CU) that are recursively divided. The present disclosure, for example, can be applied to a decoding apparatus of an High Efficiency Video Coding (HEVC) system and the like.

TECHNICAL FIELD

The present disclosure relates to a decoding apparatus and a decodingmethod, and a coding apparatus and a coding method and, moreparticularly, to a decoding apparatus and a decoding method, and acoding apparatus and a coding method capable of adjusting a compressionrate of a lowest limit.

BACKGROUND ART

Recently, in order to improve the coding efficiency more than that ofMPEG-4 Part 10 (Advanced Video Coding; hereinafter, referred to as AVC),standardization of an coding system called High Efficiency Video Coding(HEVC) has been progressed by Joint Collaboration Team-Video Coding(JCTVC) that is a joint standards organization of InternationalTelecommunication Union Telecommunication Standardization Sector (ITU-T)and International Organization for Standardization/InternationalElectrotechnical Commission (ISO/IEC) (for example, see Patent Document1).

In addition, in the HEVC, a range extension has been reviewed, forexample, for supporting a high-end dedicated format such as an image ofa color difference signal format called 4:2:2 or 4:4:4 or a profile fora screen content (for example, see Non-Patent Document 2).

CITATION LIST Non-Patent Document

-   Non-Patent Document 1: Benjamin Bross, Gary J. Sullivan, Ye-Kui    Wang, “Editors' proposed corrections to HEVC version 1”,    JCTVC-M0432_v3, 2013.425-   Non-Patent Document 2: David Flynn, Joel Sole, Teruhiko Suzuki,    “High Efficiency Video Coding (HEVC), Range Extension text    specification: Draft 4”, JCTVC-N1005_v1, 2013.8.8

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, in the HEVC system, as items defining restrictions of tools(elemental technology) necessary for decoding defined in each profile, alevel and a tier are present. The level defines a maximum value of thesize (resolution) of an image that is a coding target, and the tierdefines a maximum value of the bit rate of an image that is a codingtarget. In the HEVC system, since many images having the same size anddifferent bit rates are handled, necessary tools for decoding aredefined by using two axes of the level and the tier. In addition, foreach level, MinCr representing a compression rate of a lowest limit isset.

However, adaptation to a coded stream by adjusting the compression rateof a lowest limit has not been considered.

The present disclosure is in consideration of such situations andenables adjustment of the compression rate of a lowest limit.

Solutions to Problems

A decoding apparatus according to a first aspect of the presentdisclosure is a decoding apparatus including a decoding unit thatdecodes a bit stream coded according to a coding standard having aprofile in which a lowest compression rate at the time of coding animage is set for each of a plurality of tiers in units of blocks thatare recursively divided.

A decoding method according to the first aspect of the presentdisclosure corresponds to the decoding apparatus according to the firstaspect of the present disclosure.

According to the first aspect of the present disclosure, a bit streamcoded according to a coding standard having a profile in which a lowestcompression rate at the time of coding an image is set for each of aplurality of tiers is decoded in units of blocks that are recursivelydivided.

A coding apparatus according to a second aspect of the presentdisclosure is a coding apparatus including a coding unit that codes animage according to a coding standard having a profile in which a lowestcompression rate at the time of coding an image is set for each of aplurality of tiers in units of blocks that are recursively divided.

A coding method according to the second aspect of the present disclosurecorresponds to the coding apparatus according to the second aspect ofthe present disclosure.

According to the second aspect of the present disclosure, an image iscoded according to a coding standard having a profile in which a lowestcompression rate at the time of coding an image is set for each of aplurality of tiers in units of blocks that are recursively divided.

The decoding apparatus according to the first aspect and the codingapparatus according to the second aspect can be realized by causing acomputer to execute a program.

In order to realize the decoding apparatus according to the first aspectand the coding apparatus according to the second aspect, a program thatis executed by a computer can be provided by transmitting the programthrough a transmission medium or by recording the program on a recordingmedium.

The decoding apparatus according to the first aspect and the codingapparatus according to the second aspect may be independent apparatusesor internal blocks configuring one apparatus.

Effects of the Invention

According to the first aspect of the present disclosure, a coded streamcan be decoded. In addition, according to the first aspect of thepresent disclosure, a coded stream of which the compression rate of alowest limit is adjusted can be decoded.

According to the second aspect of the present disclosure, an image canbe coded. In addition, according to the second aspect of the presentdisclosure, the compression rate of a lowest limit can be adjusted.

The advantages described here are not necessarily limited, but there maybe included any advantage that is described in the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram that illustrates restrictions defined by a level anda tier in an HEVC system.

FIG. 2 is a diagram that illustrates unique restrictions defined by alevel and a tier in a case where a profile is Main and Main10.

FIG. 3 is a block diagram that illustrates an example of theconfiguration of a coding apparatus of a first embodiment according tothe present disclosure.

FIG. 4 is a diagram that illustrates an example of the syntax ofprofile_tier_level.

FIG. 5 is a diagram that illustrates a first example of parameters.

FIG. 6 is a diagram that illustrates a first example of parametersHbrFactor and ShFactor.

FIG. 7 is a block diagram that illustrates an example of theconfiguration of a coding unit illustrated in FIG. 3.

FIG. 8 is a diagram that illustrates a CU.

FIG. 9 is a flowchart that illustrates a stream generating processperformed by the coding apparatus illustrated in FIG. 3.

FIG. 10 is a flowchart that illustrates details of a coding processillustrated in FIG. 9.

FIG. 11 is a flowchart that illustrates details of the coding processillustrated in FIG. 9.

FIG. 12 is a block diagram that illustrates an example of theconfiguration of the decoding apparatus of the first embodimentaccording to the present disclosure.

FIG. 13 is a block diagram that illustrates an example of theconfiguration of a decoding unit illustrated in FIG. 12.

FIG. 14 is a flowchart that illustrates an image generating processperformed by the decoding apparatus illustrated in FIG. 12.

FIG. 15 is a flowchart that illustrates details of a decoding processillustrated in FIG. 14.

FIG. 16 is a diagram that illustrates a second example of parameters.

FIG. 17 is a diagram that illustrates a second example of the parametersHbrFactor and ShFactor.

FIG. 18 is a diagram that illustrates a third example of parameters.

FIG. 19 is a diagram that illustrates a third example of the parametersHbrFactor and ShFactor.

FIG. 20 is a diagram that illustrates an example of MinCr set for eachtier.

FIG. 21 is a diagram that illustrates a second embodiment according tothe present disclosure.

FIG. 22 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 23 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 24 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 25 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 26 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 27 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 28 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 29 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 30 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 31 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 32 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 33 is a diagram that illustrates the second embodiment according tothe present disclosure.

FIG. 34 is a block diagram that illustrates an example of the hardwareconfiguration of a computer.

FIG. 35 is a diagram that illustrates an example of the schematicconfiguration of a television apparatus according to the presentdisclosure.

FIG. 36 is a diagram that illustrates an example of the schematicconfiguration of a mobile phone according to the present disclosure.

FIG. 37 is a diagram that illustrates an example of the schematicconfiguration of a recording/reproducing apparatus according to thepresent disclosure.

FIG. 38 is a diagram that illustrates an example of the schematicconfiguration of an imaging apparatus according to the presentdisclosure.

FIG. 39 illustrates an example of the schematic configuration of a videoset according to the present disclosure.

FIG. 40 illustrates an example of the schematic configuration of a videoprocessor according to the present disclosure.

FIG. 41 illustrates another example of the schematic configuration of avideo processor according to the present disclosure.

MODE FOR CARRYING OUT THE INVENTION Overview of Present Disclosure(Description of Tier and Level)

FIG. 1 is a diagram that illustrates restrictions defined by a level anda tier in an HEVC system, and FIG. 2 is a diagram that illustratesunique restrictions defined by a level and a tier in a case where aprofile is Main and Main10.

As illustrated in FIGS. 1 and 2, in the HEVC system, for each level, amaximum size (Max luma picture size) of a coding target image isdefined. In addition, for each level of level 4 or a higher level,information (Max CPB size, MaxBR) representing a maximum value of thebit rate that is different for each a tier within the same level isdefined. As such tiers, there are a main tier (Main tier) defining arelatively low bit rate used for an application such as a broadcast anda high tier (High tier) defining a relatively high bit rate used for anapplication such as video production or recording on a storage medium.In addition, as illustrated in FIG. 2, MinCr is set for each level.

However, in the high tier, since the bit rate is high, there are caseswhere the compression rate of a coded stream is higher than MinCr. Insuch cases, it is necessary to perform coding again such that thecompression rate is lower than MinCr. Arithmetic coding (CABAC) using acontext used in the HEVC standard is performed while the state of theperiphery is stored as a context. Accordingly, in a case where retry isrequired in a coding unit level, it is necessary to perform retry withall the context states being stored. Thus, particularly in case of theCABAC, a calculation load accompanying the arithmetic coding accordingto the retry is high, and a load applied to the system is high.

In addition, in the HEVC system, the number of tiers is two including amain tier and a high tier, and a coded stream having a higher bit ratecannot be generated. However, in a case where an application performingvisually lossless compression, lossless compression, or low compressionfor a medical image or the like is considered, a tier corresponding to abit rate higher than the bit rate of the high tier is necessary.

Thus, in the present disclosure, identification data used foridentifying the adjustment of a compression rate of a lowest limit and amaximum value of the bit rate corresponding to a tier is set, and MinCrand the maximum value of the bit rate are adjusted based on theidentification data.

First Embodiment Example of Configuration of Coding Apparatus Accordingto First Embodiment

FIG. 3 is a block diagram that illustrates an example of theconfiguration of a coding apparatus of a first embodiment according tothe present disclosure.

The coding apparatus 10 illustrated in FIG. 3 is configured by a settingunit 11, a coding unit 12, a control unit 13, and a transmission unit 14and codes an image using a system compliant to the HEVC system.

More specifically, the setting unit 11 of the coding apparatus 10 setsparameter sets such as an Sequence Parameter Set (SPS), a PictureParameter Set (PPS), Video Usability Information (VUI), and SupplementalEnhancement. Information (SEI). To the SPS and the VPS,profile_tier_level including identification data, informationrepresenting a level and a tier, and the like are set. The setting unit11 supplies the set parameter set to the coding unit 12 and the controlunit 13.

To the coding unit 12, images in units of frames are input. The codingunit 12 performs coding of input images using the HEVC system under thecontrol of the control unit 13. The coding unit 12 generates a codedstream based on coded data acquired as a result of the coding processand a parameter set supplied from the setting unit 11 and supplies thegenerated coded stream to the transmission unit 14.

The control unit 13 calculates a compression rate of a lowest limitbased on MinCr and identification data corresponding to a levelrepresented by the information included in profile_tier_level suppliedfrom the setting unit 11. In addition, the control unit 13 calculates amaximum value of the bit rate based on a maximum value of the bit rateand identification data corresponding to the tier represented by theinformation included in profile_tier_level. The control unit 13 controlsa coding process performed by the coding unit 12 based on thecompression rate of the lowest limit and the maximum value of the bitrate.

More specifically, the control unit 13 determines whether or not thecompression rate of the coded data is higher than the compression rateof the lowest limit and, in a case where the compression rate of thecoded data is the compression rate of the lowest limit or lower, causesthe coding process to be performed again such that the compression rateof the coded data is higher than the compression rate of the lowestlimit.

The transmission unit 14 transmits a coded stream supplied from thecoding unit 12 to a decoding apparatus to be described later.

(Example of Syntax of Profile_Tier_Level)

FIG. 4 is a diagram that illustrates an example of the syntax ofprofile_tier_level.

As illustrated in FIG. 4, in profile_tier_level, a tier flag(general_tier_flag) representing whether a tier is a main tier or a hightier is set. In addition, in profile_tier_level, a low bit rate flag(general_lower_bit_rate_constraint_flag) representing whether or not thebit rate is low and identification data(general_higher_bit_rate_indication_flag) are set.

The low bit rate flag is “1” in a case where it represents that the bitrate is low and is “0” in a case where it represents that the bit rateis not low. In addition, the identification data is “1” in a case whereit represents that the maximum value of the bit rate corresponding tothe compression rate of the lowest limit and the tier is adjusted and is“0” in a case where it represents no adjustment thereof.

In addition, in profile_tier_level, level information(general_level_idc) representing a level is set.

(First Example of Parameter Used for Calculating Maximum Value of BitRate and Compression Rate of Lowest Limit)

FIG. 5 is a diagram that illustrates a first example of parameters usedfor calculating a maximum value of the bit rate and a compression rateof a lowest limit.

The maximum value of the bit rate is calculated using the followingEquation (3) by using parameters CpbBrVclFactor and CpbBrNalFactor.

[Mathematical Formula 1]

VCL HRD parameter:Max bit rate=CpbBrVclFactor*MaxBR

NAL HRD parameter:Max bit rate=CpbBrNalFactor*MaxBR   (3)

In Equation (3), Maxbitrate is a maximum value of the bit rate, andMaxBR is information representing the maximum value of the bit rate,which is illustrated in FIG. 2, corresponding to the level informationand the tier flag included in profile_tier_level.

In addition, the compression rate of the lowest limit is calculatedusing the following Equation (4) by using a parameter MinCrScaleFactor.

[Mathematical Formula 2]

MinCr=Max(1,MinCrBase*MinCrScalefactor)  (4)

In Equation (4), MinCr is the compression rate of the lowest limit, andMinCrBase is MinCr, which is represented in FIG. 2, corresponding to thelevel information and the tier flag included in profile_tier_level.

The parameter CpbBrVclFactor or CpbBrNalFactor used for the calculationof the maximum value of the bit rate and the parameter MinCrScaleFactorused for the calculation of the compression rate of the lowest limit, asillustrated in FIG. 5, are set for each profile.

In the example illustrated in FIG. 5, the parameters CpbBrVclFactor andCpbBrNalFactor are adjusted by using a parameter HbrFactor and aparameter ShFactor, and the parameter MinCrScaleFactor is adjusted byusing the parameter HbrFactor.

The parameter HbrFactor is defined by using the following Equation (5)using a low bit rate flag (general_lower_bit_rate_constraint_flag) andidentification data (general_higher_bit_rate_indication_flag) includedin profile_tier_level.

[Mathematical Formula 3]

HbrFactor=2−general_lower_bit_rate_constraint_flag+2*general_higher_bit_rate_indication_flag  (5)

The parameter ShFactor is defined by using the following Equation (6)using a low bit rate flag (general_lower_bit_rate_constraint_flag) andthe identification data (general_higher_bit_rate_indication_flag)included in profile_tier_level.

[Mathematical Formula 4]

ShFactor=1+(!general_lower_bit_rate_constraint_flag)*3*general_higher_bit_rate_indication_flag  (6)

(First Example of Parameters HbrFactor and ShFactor)

FIG. 6 is a diagram that illustrates an example of the parametersHbrFactor and ShFactor used for adjusting the parameters illustrated inFIG. 5 in a case where the profiles are Long Gop profiles and All Intraprofiles.

Here, a super high tier is a virtual tier having a maximum value of thebit rate to be higher than that of the high tier.

As illustrated in FIG. 6, in a case where the profile is Long Gopprofiles, and the tier is the main tier, a tier flag (general_tier_flag)is set to zero. In addition, a low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “1”, andidentification data (general_higher_bit_rate_indication_flag) is set to“0”. Accordingly, the parameter HbrFactor becomes “1”, and the parameterShFactor becomes “1”.

As a result, the maximum value of the bit rate and the compression rateof the lowest limit respectively become a maximum value of the bit ratecorresponding to the main tier and a compression rate of the lowestlimit represented by MinCr, and the maximum value of the bit rate andthe compression rate of the lowest limit are not adjusted.

On the other hand, in a case where the profile is the Long Gop profiles,and the tier is the high tier, the tier flag (general_tier_flag) is setto “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “1”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “2”, and the parameterShFactor becomes “1”.

As a result, while the maximum value of the bit rate becomes a maximumvalue of the bit rate corresponding to the main tier and is notadjusted, the compression rate of the lowest limit becomes ½ of thecompression rate of the lowest limit represented by MinCr.

In addition, in a case where the profile is the Long Gop profiles, andthe tier is the super high tier, the tier flag (general_tier_flag) isset to “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “0”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “4”, and the parameterShFactor becomes “4”.

As a result, the maximum value of the bit rate becomes four times thebit rate corresponding to the main tier, and the compression rate of thelowest limit becomes ¼ of the compression rate of the lowest limitrepresented by MinCr.

In a case where the profile is All Intra profiles, and the tier is themain tier, the tier flag (general_tier_flag) is set to “0”. In addition,the low bit rate flag (general_lower_bit_rate_constraint_flag) is set to“0” or “1”, and the identification data(general_higher_bit_rate_indication_flag) is set to “0”. Thus, theparameter HbrFactor becomes “2” or “1”, and the parameter ShFactorbecomes “2” or “1”.

As a result, the maximum value of the bit rate becomes twice the bitrate corresponding to the main tier, and the compression rate of thelowest limit becomes ½ of the compression rate of the lowest limitrepresented by MinCr, or both thereof are not adjusted.

On the other hand, in a case where the profile is the All Intraprofiles, and the tier is the high tier, the tier flag(general_tier_flag) is set to “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “0” or “1”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “4” or “2”, and the parameterShFactor becomes “4” or “2”.

As a result, the maximum value of the bit rate becomes four times or twotimes the bit rate corresponding to the main tier, and the compressionrate of the lowest limit becomes ¼ or ½ of the compression rate of thelowest limit represented by MinCr.

In addition, in a case where the profile is the All Intra profiles, andthe tier is the super high tier, the tier flag (general_tier_flag) isset to “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “0”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “4”, and the parameterShFactor becomes “4”.

As a result, the maximum value of the bit rate becomes four times thebit rate corresponding to the main tier, and the compression rate of thelowest limit becomes ¼ of the compression rate of the lowest limitrepresented by MinCr.

(Example of Configuration of Coding Unit)

FIG. 7 is a block diagram that illustrates an example of theconfiguration of the coding unit 12 illustrated in FIG. 3.

The coding unit 12 illustrated in FIG. 7 includes: an A/D conversionunit 31; a screen rearranging buffer 32; a calculation unit 33; anorthogonal transform unit 34; a quantization unit 35; a reversiblecoding unit 36; an accumulation buffer 37; a generation unit 38; aninverse quantization unit 39; an inverse orthogonal transform unit 40;and an addition unit 41. In addition, the coding unit 12 includes: ade-blocking filter 42; an adaptive offset filter 43; an adaptive loopfilter 44; a frame memory 45; a switch 46; an intra prediction unit 47;a motion predicting/compensating unit 48; a predicted image selectingunit 49; and a rate control unit 50.

The A/D conversion unit 31 of the coding unit 12 performs an A/Dconversion of images configured in units of frames input as codingtargets. The A/D conversion unit 31 outputs an image that is a digitalsignal after the conversion to the screen rearranging buffer 32 so as tobe stored therein.

The screen rearranging buffer 32 rearranges stored images configured inunits of frames, which are arranged in order of display, in order forcoding in accordance with a GOP structure. The screen rearranging buffer32 outputs the images after the rearrangement to the calculation unit33, the intra prediction unit 47, and the motion predicting/compensatingunit 48.

The calculation unit 33 subtracts a predicted image supplied from thepredicted image selecting unit 49 from an image supplied from the screenrearranging buffer 32, thereby performing coding. The calculation unit33 outputs an image acquired as a result thereof to the orthogonaltransform unit 34 as residual information. In addition, in a case wherea predicted image is not supplied from the predicted image selectingunit 49, the calculation unit 33 outputs an image read from the screenrearranging buffer 32 as it is to the orthogonal transform unit 34 asthe residual information.

The orthogonal transform unit 34 performs an orthogonal transform ofresidual information supplied from the calculation unit 33 in units oftransform units (TU). The orthogonal transform unit 34 suppliesorthogonal transform coefficients acquired as a result of the orthogonaltransform to the quantization unit 35.

The quantization unit 35 quantizes the orthogonal transform coefficientssupplied from the orthogonal transform unit 34. The quantization unit 35supplies the quantized orthogonal transform coefficients to thereversible coding unit 36.

The reversible coding unit 36 acquires intra prediction mode informationrepresenting an optimal intra prediction mode from the intra predict ionunit 47. In addition, the reversible coding unit 36 acquires interprediction mode information representing an optimal inter predictionmode, a motion vector, information specifying a reference image, and thelike from the motion predicting/compensating unit 48.

In addition, the reversible coding unit 36 acquires offset filterinformation relating to an offset filter from the adaptive offset filter43 and acquires filter coefficients from the adaptive loop filter 44.

The reversible coding unit 36 performs reversible coding such asvariable length coding (for example, Context-Adaptive Variable LengthCoding (CAVLC), arithmetic coding (for example, Context-Adaptive BinaryArithmetic Coding (CABAC), or the like for the quantized orthogonaltransform coefficients supplied from the quantization unit 35.

The reversible coding unit 36 performs reversible coding using intraprediction mode information or the interprediction mode information, themotion vector, the information specifying a reference image, the offsetfilter information, and the filter coefficients as coding informationrelating to the coding. The reversible coding unit 36 supplies thecoding information and the orthogonal transform coefficients, which havebeen reversibly coded, to the accumulation buffer 37 as coded data so asto be stored therein. In addition, the coding information that has beenreversibly coded may be added to the coded data as a header section suchas a slice header.

The accumulation buffer 37 temporarily stores the coded data suppliedfrom the reversible coding unit 36. In addition, the accumulation buffer37 supplies the stored coded data to the generation unit 38.

The generation unit 38 generates a coded stream based on the parametersets supplied from the setting unit 11 illustrated in FIG. 3 and thecoded data supplied from the accumulation buffer 37 and supplies thegenerated coded stream to the transmission unit 14 illustrated in FIG.3.

In addition, the quantized orthogonal transform coefficients output fromthe quantization unit 35 are input also to the inverse quantization unit39. The inverse quantization unit 39 performs inverse quantization ofthe orthogonal transform coefficients quantized by the quantization unit35 using a method corresponding to the quantization method used by thequantization unit 35. The inverse quantization unit 39 supplies theorthogonal transform coefficients acquired as a result of the inversequantization process to the inverse orthogonal transform unit 40.

The inverse orthogonal transform unit 40 performs an inverse orthogonaltransform of the orthogonal transform coefficients supplied from theinverse quantization unit 39 in units of TUs by using a methodcorresponding to the orthogonal transform method used by the orthogonaltransform unit 34. The inverse orthogonal transform unit 40 suppliesresidual information acquired as a result thereof to the addition unit41.

The addition unit 41 adds residual information supplied from the inverseorthogonal transform unit 40 and a predicted image supplied from thepredicted image selecting unit 49 and locally performs decoding. In acase where a predicted image is not supplied from the predicted imageselecting unit 49, the addition unit 41 sets the residual informationsupplied from the inverse orthogonal transform unit 40 as alocally-decoded image. The addition unit 41 supplies the locally-decodedimage to the de-blocking filter 42 and the frame memory 45.

The de-blocking filter 42 performs a de-blocking filter process ofeliminating a block distortion for the locally-decoded image suppliedfrom the addition unit 41 and supplies an image acquired as a resultthereof to the adaptive offset filter 43.

The adaptive offset filter 43 performs an adaptive offset filter (SampleAdaptive Offset (SAO)) process for mainly eliminating ringing for animage after the de-blocking filter process performed by the de-blockingfilter 42.

More specifically, the adaptive offset filter 43 determines the type ofan adaptive offset filter process for each Largest Coding Unit (LCU)that is a maximum coding unit and acquires an offset used in theadaptive offset filter process. The adaptive offset filter 43 performsthe adaptive offset filter process of the determined type for the imageafter the de-blocking filter process by using the acquired offset.

The adaptive offset filter 43 supplies the image after the adaptiveoffset filter process to the adaptive loop filter 44. In addition, theadaptive offset filter 43 supplies information representing the type ofthe performed adaptive offset filter process and the offset to thereversible coding unit 36 as offset filter information.

The adaptive loop filter 44, for example, is configured by atwo-dimensional Wiener filter. The adaptive loop filter 44, for example,performs an adaptive loop filter (Adaptive Loop Filter (ALF)) processfor the image after the adaptive offset filter process supplied from theadaptive offset filter 43 for each LCU.

More specifically, the adaptive loop filter 44, for each LCU, calculatesfilter coefficients used in the adaptive loop filter process such that aresidual between an original image that is an image output from thescreen rearranging buffer 32 and an image after the adaptive loop filteris minimal. Then, the adaptive loop filter 44 performs an adaptive loopfilter process for the image after the adaptive offset filter process byusing the calculated filter coefficients for each LCU.

The adaptive loop filter 44 supplies the image after the adaptive loopfilter process to the frame memory 45. In addition, the adaptive loopfilter 44 supplies the filter coefficients used in the adaptive loopfilter process to the reversible coding unit 36.

Here, while the adaptive loop filter process is described to beperformed for each LCU, the processing unit of the adaptive loop filterprocess is not limited to the LCU. However, by matching the processingunits of the adaptive offset filter 43 and the adaptive loop filter 44,the process can be efficiently performed.

The frame memory 45 accumulates images supplied from the adaptive loopfilter 44 and images supplied from the addition unit 41. In an image,for which a filter process has not been performed, accumulated in theframe memory 45, pixels adjacent to a Prediction Unit (PU) are suppliedto the intra prediction unit 47 through the switch 46 as surroundingpixels. On the other hand, an image, for which the filter process hasbeen performed, accumulated in the frame memory 45 is output to themotion predicting/compensating unit 48 through the switch 46 as areference image.

The intra prediction unit 47 performs an intra prediction process of allthe intra prediction modes that are candidates by using the surroundingpixels read from the frame memory 45 through the switch 46 in units ofPUs.

In addition, the intra prediction unit 47 calculates cost functionvalues (details thereof will be described later) for all the intraprediction modes that are the candidates based on the image read fromthe screen rearranging buffer 32 and the predicted image generated as aresult of the intra prediction process. Then, the intra prediction unit47 determines an intra prediction mode of which the cost function valueis minimal as an optimal intra prediction mode.

The intra prediction unit 47 supplies a predicted image generated in theoptimal intra prediction mode and a corresponding cost function value tothe predicted image selecting unit 49. In a case where the selection ofa predicted image generated in the optimal intra prediction mode isnotified from the predicted image selecting unit 49, the intraprediction unit 47 supplies the intra prediction mode information to thereversible coding unit 36.

In addition, the cost function value is also called an Rate Distortion(RD) cost and, for example, is calculated based on a technique of a highcomplexity mode or a low complexity mode as defined in a Joint Model(JM) that is reference software in an H.264/AVC system. The referencesoftware in the H.264/AVC system is disclosed inhttp://iphome.hhi.de/suehring/tml/index.htm.

More specifically, in a case where the high complexity mode is employedas a technique for calculating the cost function value, a cost functionvalue represented in the following Equation (1) is calculated for eachprediction mode by provisionally performing up to decoding for all theprediction modes that are candidates.

[Mathematical Formula 5]

Cost(Mode)=D+λ·R  (1)

Here, D is a difference (distortion) between the original image and adecoded image, R is a generated coding amount including up to theorthogonal transform coefficients, and λ is a Lagrange undefinedmultiplier given as a function of a quantization parameter (QP).

On the other hand, in a case where the low complexity mode is employedas a technique for calculating the cost function value, a cost function(Cost(Mode)) represented in the following Equation (2) is calculated foreach prediction mode by generating a predicted image and calculating thecoding amount of the coding information for all the prediction modesthat are candidates.

[Mathematical Formula 6]

Cost(Mode)=D+QPtoQuant(QP)·Header_Bit  (2)

Here, D is a difference (distortion) between the original image and apredicted image, Header_Bit is a coding amount of the codinginformation, and QPtoQuant is a function given as a function of aquantization parameter QP.

In the low complexity mode, only predicted images may be generated forall the prediction modes, and a decoded image does not need to begenerated, and the amount of calculation is small.

The motion predicting/compensating unit 48 performs motionpredicting/compensating processes of all the inter prediction modes thatare candidates in units of PUs. More specifically, the motionpredicting/compensating unit 48 includes a two-dimensional linearinterpolation adaptive filter. In addition, the motionpredicting/compensating unit 48 performs an interpolation filter processfor an image supplied from the screen rearranging buffer 32 and areference image by using the two-dimensional linear interpolationadaptive filter, thereby increasing the resolutions of the image and thereference image.

The motion predicting/compensating unit 48 detects motion vectors of allthe inter prediction modes that are candidates with fractional pixelaccuracy based on the image and the reference image of which theresolutions have been increased. Then, the motionpredicting/compensating unit 48 performs a compensation process for thereference image based on the motion vector, thereby generating apredicted image. Here, the inter prediction mode is a mode thatrepresents the size of the PU and the like.

In addition, the motion predicting/compensating unit 48 calculates costfunction values for all the inter prediction modes that are candidatesbased on the image supplied form the screen rearranging buffer 32 andthe predicted image and determines an inter prediction mode of which thecost function value is minimal as an optimal inter prediction mode.Then, the motion predicting/compensating unit 48 supplies the costfunction value of the optimal inter prediction mode and a correspondingpredicted image to the predicted image selecting unit 49.

In addition, in a case where the selection of the predicted imagegenerated in the optimal inter prediction mode is notified from thepredicted image selecting unit 49, the motion predicting/compensatingunit 48 outputs the inter prediction mode information, a correspondingmotion vector, the information specifying the reference image, and thelike to the reversible coding unit 36.

The predicted image selecting unit 49 determines one of the optimalintra prediction mode and the optimal inter prediction mode that has asmaller corresponding cost function value as an optimal prediction modebased on the cost function values supplied from the intra predictionunit 47 and the motion predicting/compensating unit 48. Then, thepredicted image selecting unit 49 supplies the predicted image of theoptimal prediction mode to the calculation unit 33 and the addition unit41. In addition, the predicted image selecting unit 49 notifies theintra prediction unit 47 or the motion predicting/compensating unit 48of the selection of the predicted image of the optimal prediction mode.

The rate control unit 50 controls the rate of the quantization operationperformed by the quantization unit 35 based on the coded dataaccumulated in the accumulation buffer 37 such that an overflow or anunderflow does not occur.

(Description of Coding Unit)

FIG. 8 is a diagram that illustrates a coding unit (CU) that is a codingunit used in the HEVC system.

The HEVC system is targeted also for an image having a large pictureframe such as Ultra High Definition (UHD) of 4000 pixels×2000 pixels,and thus, it is not optimal to fix the size of the coding unit to 16pixels×16 pixels. Accordingly, in the HEVC system, a CU is defined as acoding unit. Details of the CU are described in Non-Patent Document 1.

The CU (coding block) achieves the same role as that of a macro block inthe AVC system. More specifically, the CU is divided into PUs or dividedinto TUs.

However, the CU has a square shape of which the size is represented bypixels of exponent of two that is changeable for each sequence. Morespecifically, the CU is set by recursively dividing an LCU, which is aCU having a maximum size, into two parts for an arbitrary number oftimes so as not to be less than an Smallest Coding Unit (SCU) that is aCU having a minimal size in the horizontal direction and the verticaldirection. In other words, the size of an arbitrary hierarchy at thetime of hierarchizing an LCU such that the size of a higher hlerarchy is¼ of the size of a lower hierarchy until the LCU becomes an SCU is thesize of the CU.

For example, in the case illustrated in FIG. 8, the size of the LCU is128, and the size of the SCU is 8. Thus, the hierarchical depths (Depth)of the LCU are 0 to 4, and the number of the hierarchical depths is 5.In other words, the number of divisions corresponding to the CU is oneof “0” to “4”.

Here, information designating the sizes of the LCU and the SCU isincluded in the SPS. In addition, the number of divisions correspondingto the CU is designated by split_flag representing whether or notfurther division is performed for each hierarchy.

The size of the TU, similarly to split_flag of the CU, can be designatedusing split_transform_flag. The maximum numbers of divisions of the TUat the time of performing an inter prediction and at the time ofperforming an intra prediction are respectively designated asmax_transform_hierarchy_depth_inter andmax_transform_hierarchy_depth_intra by using the SPS.

In this specification, a Coding Tree Unit (CTU) is assumed to be a unitthat includes a Coding Tree Block (CTB) of a LCU and a parameter at thetime of processing at the LCU base (level) thereof. In addition, a CUconfiguring the CTU is assumed to be a unit that includes a Coding Block(CB) and a parameter at the time of processing at the CU base (level)thereof.

(Description of Process Performed by Coding Apparatus)

FIG. 9 is a flowchart that illustrates a stream generating processperformed by the coding apparatus 10 illustrated in FIG. 3.

As illustrated in FIG. 9, in Step S11, the setting unit 11 of the codingapparatus 10 sets a parameter set. The setting unit 11 supplies the setparameter set to the coding unit 12 and the control unit 13.

In Step S12, the control unit 13 calculates a compression rate of alowest limit based on MinCr and identification data corresponding to alevel represented by information included in profile_tier_level suppliedfrom the setting unit 11. In addition, the control unit 13 calculates amaximum value of the bit rate based on a maximum value of the bit rateand identification data corresponding to a tier represented byinformation included in profile_tier_level. The control unit 13 controlsthe coding process of Step S13 based on the compression rate of thelowest limit and the maximum value of the bit rate.

In Step S13, the coding unit 12 performs a coding process of codingimages configured in units of frames input from the outside by using theHEVC system. Details of this coding process will be described withreference to FIGS. 10 and 11 to be described later.

In Step S14, the generation unit 38 (FIG. 7) of the coding unit 12generates a coded stream based on the parameter set supplied from thesetting unit 11 and the accumulated coded data and supplies thegenerated coded stream to the transmission unit 14.

In Step S15, the transmission unit 14 transmits the coded streamsupplied from the setting unit 11 to the decoding apparatus to bedescribed later and ends the process.

FIGS. 10 and 11 represent a flowchart that illustrates details of thecoding process of Step S13 illustrated in FIG. 9.

In Step S31 illustrated in FIG. 10, the A/D conversion unit 31 of thecoding unit 12 performs an A/D conversion of images configured in unitsof frames input as a coding target. The A/D conversion unit 31 outputsan image that is a digital signal after the conversion to the screenrearranging buffer 32 so as to be stored therein.

In Step S32, the screen rearranging buffer 32 rearranges the storedimages of frames in order of display into order for coding in accordancewith a GOP structure. The screen rearranging buffer 32 supplies theimages configured in units of frames after the rearrangement to thecalculation unit 33, the intra prediction unit 47, and the motionpredicting/compensating unit 48.

In Step S33, the intra prediction unit 47 performs intra predictionprocesses of all the intra prediction modes that are candidates in unitsof PUs. In addition, the intra prediction unit 47 calculates costfunction values for all the intra prediction modes that are candidatesbased on images read from the screen rearranging buffer 32 and predictedimages generated as a result of the intra prediction processes. Then,the intra prediction unit 47 determines an intra prediction mode ofwhich the cost function value is minimal as an optimal intra predictionmode. The intra prediction unit 47 supplies a predicted image generatedin the optimal intra prediction mode and a corresponding cost functionvalue to the predicted image selecting unit 49.

In addition, the motion predicting/compensating unit 48 performs motionpredicting/compensating processes of all the inter prediction modes thatare candidates in units of PUs. In addition, the motionpredicting/compensating unit 48 calculates cost function values for allthe inter prediction modes that are candidates based on the imagessupplied from the screen rearranging buffer 32 and predicted images anddetermines an inter prediction mode of which the cost function value isminimal as an optimal inter prediction mode. Then, the motionpredicting/compensating unit 48 supplies the cost function value of theoptimal inter prediction mode and a corresponding predicted image to thepredicted image selecting unit 49.

In Step S34, the predicted image selecting unit 49, based on the costfunction values supplied from the intra prediction unit 47 and themotion predicting/compensating unit 48, determines one of the optimalintra prediction mode and the optimal inter prediction mode of which thecost function value is minimal as an optimal prediction mode. Then, thepredicted image selecting unit 49 supplies a predicted image of theoptimal prediction mode to the calculation unit 33 and the addition unit41.

In Step S35, the predicted image selecting unit 49 determines whether ornot the optimal prediction mode is the optimal inter prediction mode. Ina case where the optimal prediction mode is determined to be the optimalinter prediction mode in Step S35, the predicted image selecting unit 49notifies the motion predicting/compensating unit 48 of the selection ofa predicted image generated in the optimal inter prediction mode.

Then, in Step S36, the motion predicting/compensating unit 48 suppliesthe inter prediction mode information, the motion vector, and theinformation specifying a reference image to the reversible coding unit36, and the process proceeds to Step S38.

On the other hand, in a case where the optimal prediction mode isdetermined not to be the optimal inter prediction mode in Step S35, inother words in a case where the optimal prediction mode is the optimalintra prediction mode, the predicted image selecting unit 49 notifiesthe intra prediction unit 47 of the selection of a predicted imagegenerated in the optimal intra prediction mode. Then, in Step S37, theintra prediction unit 47 supplies the intra prediction mode informationto the reversible coding unit 36, and the process proceeds to Step S38.

In Step S38, the calculation unit 33 performs coding by subtracting apredicted image supplied from the predicted image selecting unit 49 froman image supplied from the screen rearranging buffer 32. The calculationunit 33 outputs an image acquired as a result thereof to the orthogonaltransform unit 34 as residual information.

In Step S39, the orthogonal transform unit 34 performs an orthogonaltransform for the residual information supplied from the calculationunit 33 in units of TUs and supplies orthogonal transform coefficientsacquired as a result thereof to the quantization unit 35.

In Step S40, the quantization unit 35 quantizes the orthogonal transformcoefficients supplied form the orthogonal transform unit 34 and suppliesthe quantized orthogonal transform coefficients to the reversible codingunit 36 and the inverse quantization unit 39.

In Step S41 illustrated in FIG. 11, the inverse quantization unit 39performs inverse quantization of the quantized coefficients suppliedfrom the quantization unit 35 and supplies the orthogonal transformcoefficients acquired as a result thereof to the inverse orthogonaltransform unit 40.

In Step S42, the inverse orthogonal transform unit 40 performs aninverse orthogonal transform of the orthogonal transform coefficientssupplied from the inverse quantization unit 39 in units of TUs andsupplies residual information acquired as a result thereof to theaddition unit 41.

In Step S43, the addition unit 41 adds the residual information suppliedfrom the inverse orthogonal transform unit 40 and the predicted imagesupplied from the predicted image selecting unit 49 and locally performsdecoding. The addition unit 41 supplies the locally-decoded image to thede-blocking filter 42 and the frame memory 45.

In Step S44, the de-blocking filter 42 performs a de-blocking filterprocess for the locally-decoded image supplied from the addition unit41. The de-blocking filter 42 supplies an image acquired as a resultthereof to the adaptive offset filter 43.

In Step S45, the adaptive offset filter 43 performs an adaptive offsetfilter process for the image supplied from the de-blocking filter 42 foreach LCU. The adaptive offset filter 43 supplies an image acquired as aresult thereof to the adaptive loop filter 44. In addition, the adaptiveoffset filter 43 supplies offset filter information to the reversiblecoding unit 36 for each LCU.

In Step S46, the adaptive loop filter 44 performs an adaptive loopfilter process for the image supplied from the adaptive offset filter 43for each LCU. The adaptive loop filter 44 supplies an image acquired asa result thereof to the frame memory 45. In addition, the adaptive loopfilter 44 supplies the filter coefficients used in the adaptive loopfilter process to the reversible coding unit 36.

In Step S47, the frame memory 45 accumulates the image supplied from theadaptive loop filter 44 and the image supplied from the addition unit41. In an image, for which a filter process has not been performed,accumulated in the frame memory 45, pixels adjacent to a PU are suppliedto the intra prediction unit 47 through the switch 46 as surroundingpixels. On the other hand, an image, for which the filter process hasbeen performed, accumulated in the frame memory 45 is output to themotion predicting/compensating unit 46 through the switch 46 as areference image.

In Step S48, the reversible coding unit 36 performs reversible coding ofthe intra prediction mode information or the inter prediction modeinformation, the motion vector, the information specifying a referenceimage, the offset filter information, and the filter coefficients ascoding information.

In Step S49, the reversible coding unit 36 performs reversible coding ofthe quantized orthogonal transform coefficients supplied from thequantization unit 35. Then, the reversible coding unit 36 generatescoded data based on the coding information that has been reversiblycoded in the process of Step S48 and the orthogonal transformcoefficients that have been reversibly coded and supplies the generatedcoded data to the accumulation buffer 37.

In Step S50, the accumulation buffer 37 temporarily accumulates thecoded data supplied from the reversible coding unit 36.

In Step S51, the rate control unit 50 controls the rate of thequantization operation performed by the quantization unit 35 based onthe coded data accumulated in the accumulation buffer 37 such that anoverflow or underflow does not occur.

In Step S52, the accumulation buffer 37 outputs the stored coded data tothe generation unit 38. Then, the process is returned to Step S13illustrated in FIG. 9 and proceeds to Step S14. Then, the process isreturned to Step S13 illustrated in FIG. 9 and proceeds to Step S14.

In the coding process illustrated in FIGS. 10 and 11, for thesimplification of description, while the intra prediction process andthe motion predicting/compensating process are configured to beperformed all the time, actually, there are also cases where only onethereof is performed based on the picture type or the like.

As above, since the identification data is set, the coding apparatus 10can adjust the compression rate of a lowest limit and the maximum valueof the bit rate based on the identification data. As a result, a retrialof the coding process can be suppressed. In addition, a coded stream ofa higher bit rate can be generated.

(Example of Configuration of Decoding Apparatus According to FirstEmbodiment)

FIG. 12 is a block diagram that illustrates an example of theconfiguration of a decoding apparatus of the first embodiment accordingto the present disclosure that decodes a coded stream transmitted fromthe coding apparatus 10 illustrated in FIG. 3.

The decoding apparatus 110 illustrated in FIG. 12 is configured by: areception unit 111; an extraction unit 112; a control unit 113; and adecoding unit 114.

The reception unit 111 of the decoding apparatus 110 receives a codedstream transmitted from the coding apparatus 10 illustrated in FIG. 3and supplies the received coded stream to the extraction unit 112.

The extraction unit (parsing unit) 112 extracts (parses) a parameter setand coded data from the coded stream supplied from the reception unit111 and supplies the extracted coded data to the decoding unit 114. Inaddition, the extraction unit 112 supplies the parameter set to thecontrol unit 113.

The control unit 113 calculates a compression rate of a lowest limit,similarly to the control unit 13 illustrated in FIG. 3, based on MinCrand identification data corresponding to a level represented by theinformation included in profile_tier_level supplied from the extractionunit 112. In addition, the control unit 113 calculates a maximum valueof the bit rate, similarly to the control unit 13, based on a maximumvalue of the bit rate and identification data corresponding to the tierrepresented by the information included in profile_tier_level. Thecontrol unit 113 controls the decoding process performed by the decodingunit 114 based on the compression rate of the lowest limit and themaximum value of the bit rate.

The decoding unit 114 decodes the coded data supplied from theextraction unit 112 in units of CUs by using the HEVC system under thecontrol of the control unit 113. At this time, the decoding unit 114, asis necessary, refers to the parameter set supplied from the extractionunit 112. The decoding unit 114 outputs an image acquired as a result ofthe decoding process.

(Example of Configuration of Decoding Unit)

FIG. 13 is a block diagram that illustrates an example of theconfiguration of the decoding unit 114 illustrated in FIG. 12.

The decoding unit 114 illustrated in FIG. 13 includes: an accumulationbuffer 131; a reversible decoding unit 132; an inverse quantization unit133; an inverse orthogonal transform unit 134; an addition unit 135; ade-blocking filter 136; an adaptive offset filter 137; an adaptive loopfilter 138; and a screen rearranging buffer 139. In addition, thedecoding unit 114 includes: a D/A conversion unit 140; a frame memory141; a switch 142; an intra prediction unit 143; a motion compensationunit 144; and a switch 145.

The accumulation buffer 131 of the decoding unit 114 receives coded datafrom the extraction unit 112 illustrated in FIG. 12 and accumulates thereceived coded data. The accumulation buffer 131 supplies theaccumulated coded data to the reversible decoding unit 132.

The reversible decoding unit 132 performs reversible decoding such asvariable length decoding or arithmetic decoding corresponding to thereversible coding process performed by the reversible coding unit 36illustrated in FIG. 7 for the coded data supplied from the accumulationbuffer 131, thereby acquiring the quantized orthogonal transformcoefficients and the coding information. The reversible decoding unit132 supplies the quantized orthogonal transform coefficients to theinverse quantization unit 133. In addition, the reversible decoding unit132 supplies the intra prediction mode information and the like to theintra prediction unit 143 as coding information. The reversible decodingunit 132 supplies the motion vector, the inter prediction modeinformation, the information specifying a reference image, and the liketo the motion compensation unit 144.

Furthermore, the reversible decoding unit 132 supplies the intraprediction mode information or the inter prediction mode information ascoding information to the switch 145. The reversible decoding unit 132supplies the offset filter in formation as coding information to theadaptive offset filter 137. The reversible decoding unit 132 suppliesthe filter coefficients as coding information to the adaptive loopfilter 138.

The inverse quantization unit 133, the inverse orthogonal transform unit134, the addition unit 135, the de-blocking filter 136, the adaptiveoffset filter 137, the adaptive loop filter 138, the frame memory 141,the switch 142, the intra prediction unit 143, and the motioncompensation unit 144 respectively perform processes similar to theprocesses performed by the inverse quantization unit 39, the inverseorthogonal transform unit 40, the addition unit 41, the de-blockingfilter 42, the adaptive offset filter 43, the adaptive loop filter 44,the frame memory 45, the switch 46, the intra prediction unit 47, andthe motion predicting/compensating unit 48 illustrated in FIG. 7, andaccordingly, an image is decoded.

More specifically, the inverse quantization unit 133 performs inversequantization of the quantized orthogonal transform coefficients suppliedfrom the reversible decoding unit 132 and supplies orthogonal transformcoefficients acquired as a result thereof to the inverse orthogonaltransform unit 134.

The inverse orthogonal transform unit 134 performs an inverse orthogonaltransform of the orthogonal transform coefficients supplied from theinverse quantization unit 133 in units of TUs. The inverse orthogonaltransform unit 134 supplies residual information acquired as a result ofthe inverse orthogonal transform to the addition unit 135.

The addition unit 135 adds the residual information supplied from theinverse orthogonal transform unit 134 and the predicted image suppliedfrom the switch 145, thereby performing decoding. The addition unit 135supplies an image acquired as a result of the decoding process to thede-blocking filter 136 and the frame memory 141.

In addition, in a case where a predicted image is not supplied from theswitch 145, the addition unit 135 supplies an image that is the residualinformation supplied from the inverse orthogonal transform unit 134 tothe de-blocking filter 136 and the frame memory 141 as an image acquiredas a result of the decoding process.

The de-blocking filter 136 performs a de-blocking filter process for theimage supplied from the addition unit 135 and supplies an image acquiredas a result thereof to the adaptive offset filter 137.

The adaptive offset filter 137 performs an adaptive offset filterprocess of a type represented by the offset filter information for animage after the de-blocking filter process for each LCU by using anoffset represented by the offset filter information supplied from thereversible decoding unit 132. The adaptive offset filter 137 supplies animage after the adaptive offset filter process to the adaptive loopfilter 138.

The adaptive loop filter 138 performs an adaptive loop filter processfor an image supplied from the adaptive offset filter 137 for each LCUby using the filter coefficients supplied from the reversible decodingunit 132. The adaptive loop filter 138 supplies an image acquired as aresult thereof to the frame memory 141 and the screen rearranging buffer139.

The screen rearranging buffer 139 stores images supplied from theadaptive loop filter 138 in units of frames. The screen rearrangingbuffer 139 rearranges stored images configured in units of frames inorder for coding into the original display order and supplies therearranged images to the D/A conversion unit 140.

The D/A conversion unit 140 performs D/A conversion of the imagesconfigured in units of frames supplied from the screen rearrangingbuffer 139 and outputs converted images.

The frame memory 141 accumulates images supplied from the adaptive loopfilter 138 and images supplied from the addition unit 135. In an image,for which a filter process has not been performed, accumulated in theframe memory 141, pixels adjacent to a PU are supplied to the intraprediction unit 143 through the switch 142 as surrounding pixels. On theother hand, an image, for which the filter process has been performed,accumulated in the frame memory 141 is supplied to the motioncompensation unit 144 through the switch 142 as a reference image.

The intra prediction unit 143 performs an intra prediction process ofthe optimal intra prediction mode represented by the intra predictionmode information supplied from the reversible decoding unit 132 by usingthe surrounding pixels read from the frame memory 141 through the switch142 in units of PUs. The intra prediction unit 143 supplies a predictedimage generated as a result thereof to the switch 145.

The motion compensation unit 144 reads a reference image, which isspecified by the information specifying a reference image that issupplied from the reversible decoding unit 132, from the frame memory141 through the switch 142. The motion compensation unit 144 includes atwo-dimensional linear interpolation adaptation filter. The motioncompensation unit 144 performs an interpolation filter process for thereference image by using the two-dimensional linear interpolationadaptation filter, thereby increasing the resolution of the referenceimage. The motion compensation unit 144 performs a motion compensationprocess of the optimal inter prediction mode represented by the interprediction mode information supplied from the reversible decoding unit132 in units of PUs by using the reference image of which the resolutionhas been increased and the motion vector supplied from the reversibledecoding unit 132. The motion compensation unit 144 supplies a predictedimage generated as a result thereof to the switch 145.

In a case where the intra prediction mode information is supplied fromthe reversible decoding unit 132, the switch 145 supplies a predictedimage supplied from the intra prediction unit 143 to the addition unit135. On the other hand, in a case where the inter prediction modeinformation is supplied from the reversible decoding unit 132, theswitch 145 supplies a predicted image supplied from the motioncompensation unit 144 to the addition unit 135.

(Description of Process of Decoding Apparatus)

FIG. 14 is a flowchart that illustrates an image generating processperformed by the decoding apparatus 110 illustrated in FIG. 12.

In Step S111 illustrated in FIG. 14, the reception unit 111 of thedecoding apparatus 110 receives a coded stream transmitted from thecoding apparatus 10 illustrated in FIG. 3 and supplies the receivedcoded stream to the extraction unit 112.

In Step S112, the extraction unit 112 extracts the coded data and theparameter set from the coded stream supplied from the reception unit 111and supplies the coded data and the parameter set, which have beenextracted, to the decoding unit 114.

In Step S113, the control unit 113 calculates a compression rate of alowest limit based on MinCr and identification data corresponding to alevel, represented by the information included in profile_tier_levelsupplied from the extraction unit 112. In addition, the control unit 113calculates a maximum value of the bit rate based on the maximum value ofthe bit rate and identification data corresponding to a tier representedby the information included in profile_tier_level. The control unit 113controls the decoding process of Step S114 based on the compression rateof the lowest limit and the maximum value of the bit rate.

In Step S114, the decoding unit 114 performs a decoding process ofdecoding coded data supplied from the extraction unit 112 by using asystem compliant with the HEVC system using the parameter set suppliedfrom the extraction unit 112 as is necessary. Details of this decodingprocess will be described later with reference to FIG. 15 to bedescribed later. Then, the process ends.

FIG. 15 is a flowchart that illustrates details of the decoding processof Step S114 illustrated in FIG. 14.

In Step S131 illustrated in FIG. 15, the accumulation buffer 131 (FIG.13) of the decoding unit 114 receives coded data in units of frames fromthe extraction unit 112 illustrated in FIG. 12 and accumulates thereceived data. The accumulation buffer 131 supplies the accumulatedcoded data to the reversible decoding unit 132.

In Step S132, the reversible decoding unit 132 performs reversibledecoding of the coded data supplied from the accumulation buffer 131,thereby acquiring the quantized orthogonal transform coefficients andthe coding information. The reversible decoding unit 132 supplies thequantized orthogonal transform coefficients to the inverse quantizationunit 133.

In addition, the reversible decoding unit 132 supplies the intraprediction mode information and the like as coding information to theintra prediction unit 143. The reversible decoding unit 132 supplies themotion vector, the inter prediction mode information, the informationspecifying a reference image, and the like to the motion compensationunit 144.

In addition, the reversible decoding unit 132 supplies the intraprediction mode information or the inter prediction mode information ascoding information to the switch 145. The reversible decoding unit 132supplies the offset filter information as coding information to theadaptive offset filter 137 and supplies the filter coefficients to theadaptive loop filter 138.

In Step S133, the inverse quantization unit 133 performs inversequantization of the quantized orthogonal transform coefficients suppliedfrom the reversible decoding unit 132 and supplies orthogonal transformcoefficients acquired as a result thereof to the inverse orthogonaltransform unit 134.

In Step S134, the inverse orthogonal transform unit 134 performs aninverse orthogonal transform for the orthogonal transform coefficientssupplied from the inverse quantization unit 133 and supplies residualinformation acquired as a result thereof to the addition unit 135.

In Step S135, the motion compensation unit 144 determines whether or notthe inter prediction mode information has been supplied from thereversible decoding unit 132. In a case where the inter prediction modeinformation is determined to have been supplied in Step S135, theprocess proceeds to Step S136.

In Step S136, the motion compensation unit 144 reads a reference imagebased on the reference image specifying information supplied from thereversible decoding unit 132 in units of PUs and performs a motioncompensation process of an optimal inter prediction mode represented bythe inter prediction mode information by using the motion vector and thereference image. The motion compensation unit 144 supplies a predictedimage generated as a result thereof to the addition unit 135 through theswitch 145, and the process proceeds to Step S138.

On the other hand, in a case where the inter prediction mode informationis determined not to have been supplied in Step S135, in other words, ina case where the intra prediction mode information is supplied to theintra prediction unit 143, the process proceeds to Step S137.

In Step S137, the intra prediction unit 143 performs an intra predictionprocess of an intra prediction mode represented by the intra predictionmode information by using surrounding pixels read from the frame memory141 through the switch 142 in units of PUs. The intra prediction unit143 supplies a predicted image generated as a result of the intraprediction process to the addition unit 135 through the switch 145, andthe process proceeds to Step S138.

In Step S138, the addition unit 135 adds the residual informationsupplied from the inverse orthogonal transform unit 134 and thepredicted image supplied from the switch 145, thereby locally performingdecoding. The addition unit 135 supplies an image acquired as a resultof the decoding process to the de-blocking filter 136 and the framememory 141.

In Step S139, the de-blocking filter 136 performs a de-blocking filterprocess for the image supplied from the addition unit 135, therebyeliminating a block distortion. The de-blocking filter 136 supplies animage acquired as a result thereof to the adaptive offset filter 137.

In Step S140, the adaptive offset filter 137 performs an adaptive offsetfilter process for an image supplied from the de-blocking filter 136 foreach LCU based on the offset filter information supplied from thereversible decoding unit 132. The adaptive offset filter 137 supplies animage after the adaptive offset filter process to the adaptive loopfilter 138.

In Step S141, the adaptive loop filter 138 performs an adaptive loopfilter process for an image supplied from the adaptive offset filter 137for each LCU by using the filter coefficients supplied from thereversible decoding unit 132. The adaptive loop filter 138 supplies animage acquired as a result thereof to the frame memory 141 and thescreen rearranging buffer 139.

In Step S142, the frame memory 141 accumulates an image supplied fromthe addition unit 135 and an image supplied from the adaptive loopfilter 138. In an image, for which the filter process has not beenperformed, accumulated in the frame memory 141, pixels adjacent to thePU are supplied to the intra prediction unit 143 through the switch 142as surrounding pixels. On the other hand, an image, for which the filterprocess has been performed, accumulated in the frame memory 141 issupplied to the motion compensation unit 144 through the switch 142 as areference image.

In Step S143, the screen rearranging buffer 139 stores images suppliedfrom the adaptive loop filter 138 in units of frames, rearranges storedimages configured in units of frames, which are in order for coding, inthe original display order, and supplies the rearranged images to theD/A conversion unit 140.

In Step S144, the D/A conversion unit 140 performs a D/A conversion ofimages configured in units of frames supplied from the screenrearranging buffer 139 and outputs the converted images. Then, theprocess is returned to Step S114 illustrated in FIG. 14, and the processends.

As above, the decoding apparatus 110 adjusts the compression rate of thelowest limit and the maximum value of the bit rate based on theidentification data. Accordingly, a coded stream, which is generated bythe coding apparatus 10, of which the restrictions that are thecompression rate of the lowest limit and the maximum value of the bitrate are adjusted, can be decoded.

(Second Example of Parameter Used for Calculating Maximum Value of BitRate and Compression Rate of Lowest Limit)

FIG. 16 is a diagram that illustrates a second example of parametersused for calculating a maximum value of the bit rate and a compressionrate of a lowest limit.

In the example illustrated in FIG. 16, a parameter MinCrScaleFactor of acase where the profile is a Main profile or a Main 10 profile isadjusted by a parameter ShFactor, and the parameter ShFactor is definednot by Equation (6) described above but by the following Equation (7),which are different from the example illustrated in FIG. 5.

[Mathematical Formula 7]

ShFactor=1+(!general_lower_bit_rate_constraint_flag)*general_higher_bit_rate_indication_flag  (7)

(Second Example of Parameters HbrFactor and ShFactor)

FIG. 17 is a diagram that illustrates an example of the parametersHbrFactor and ShFactor used for adjusting the parameters illustrated inFIG. 16 in a case where the profiles are Long Gop profiles and All Intraprofiles.

Here, a super high tier is a virtual tier having a maximum value of thebit rate to be higher than that of the high tier.

As illustrated in FIG. 17, in a case where the profile is Long Gopprofiles, and the tier is the main tier, a tier flag (general_tier_flag)is set to zero. In addition, a low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “1”, andidentification data (general_higher_bit_rate_indication_flag) is set to“0”. Accordingly, the parameter HbrFactor becomes “1”, and the parameterShFactor becomes “1”.

As a result, the maximum value of the bit rate and the compression rateof the lowest limit respectively become a maximum value of the bit ratecorresponding to the main tier and a compression rate of the lowestlimit represented by MinCr, and the maximum value of the bit rate andthe compression rate of the lowest limit are not adjusted.

On the other hand, in a case where the profile is the Long Gop profiles,and the tier is the high tier, the tier flag (general_tier_flag) is setto “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “1”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “2”, and the parameterShFactor becomes “1”.

As a result, the maximum value of the bit rate becomes the maximum valueof a bit rate corresponding to the main tier and thus, is not adjusted.However, in a case where the profile is other than the Main profile andthe Main 10 profile, the compression rate of the lowest limit becomes ½of the compression rate of the lowest limit represented by MinCr. In acase where the profile is the Main profile or the Main 10 profile, thecompression rate of the lowest limit is not adjusted as well.

In addition, in a case where the profile is the Long Gop profiles, andthe tier is the super high tier, the tier flag (general_tier_flag) isset to “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “0”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “4”, and the parameterShFactor becomes “2”.

As a result, the maximum value of the bit rate becomes twice the bitrate corresponding to the main tier. In addition, in a case where theprofile is other than the Main profile and the Main 10 profile, thecompression rate of the lowest limit becomes ¼ of the compression rateof the lowest limit represented by MinCr. In a case where the profile isthe Main profile or the Main 10 profile, the compression rate of thelowest limit becomes ½ of the compression rate of the lowest limitrepresented by MinCr.

As above, in a case where the profile is the Main profile or the Main 10profile, the compression rate of the lowest limit is not adjusted at thetime of the high tier, but the compression rate of the lowest limit isadjusted only at the time of the super high tier.

A case where the profile is the All Intra profiles is similar to thecase illustrated in FIG. 6, and thus, the description thereof will notbe presented.

In the description presented above, while the identification data isused for identifying the adjustment of both the compression rate of thelowest limit and the maximum value of the bit rate corresponding to thetier, the identification data may be configured to be used foridentifying the adjustment of one thereof. In such a case, based on theidentification data, one of the compression rate of the lowest limit andthe maximum value of the bit rate corresponding to the tier is adjusted.

For example, in a case where the identification data is used foridentifying the adjustment of only the compression rate of the lowestlimit, parameters CpbBrVclFactor, CpbBrNalFactor, and MinCrScaleFactorare as illustrated in FIG. 18.

In other words, as illustrated in FIG. 18, the parameters CpbBrVclFactorand CpbBrNalFactor are adjusted using a parameter HbrFactor′, and theparameter MinCrScaleFactor is adjusted using the parameter HbrFactor.

The parameter HbrFactor′ is defined in the following Equation (8) usingthe low bit rate flag (general_lower_bit_rate_constraint_flag) includedin profile_tier_level.

[Mathematical Formula 8]

HbrFactor′=2−general_lower_bit_rate_constraint_flag   (8)

In addition, the parameter HbrFactor of a case where the profile is theLong Gop profiles and the All Intra profiles, for example, is asillustrated in FIG. 19.

As illustrated in FIG. 19, in a case where the profile is the Long Gopprofiles, and the tier is the main tier, the tier flag(general_tier_flag) is set to “0”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “1”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“0”. Thus, the parameter HbrFactor becomes “1”. As a result, thecompression rate of the lowest limit becomes the compression rate of thelowest limit represented by MinCr and thus, is not adjusted.

On the other hand, in a case where the profile is the Long Gop profiles,and the tier is the high tier, the tier flag (general_tier_flag) is setto “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “1”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “2”. As a result, thecompression rate of the lowest limit becomes ½ of the compression rateof the lowest limit represented by MinCr.

In case of the example illustrated in FIG. 18, in a case where theprofile is the Long Gop profiles, the maximum value of the bit rate isnot adjusted.

In addition, in a case where the profile is the All Intra profiles, andthe tier is the main tier, the tier flag (general_tier_flag) is set to“0”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “0” or “1”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“0”. Thus, the parameter HbrFactor becomes “2” or “1”, and the parameterShFactor becomes “2” or “1”.

As a result, the maximum value of the bit rate becomes twice the bitrate corresponding to the main tier, and the compression rate of thelowest limit becomes ½ of the compression rate of the lowest limitrepresented by MinCr, or both are not adjusted.

On the other hand, in a case where the profile is the All Intraprofiles, and the tier is the high tier, the tier flag(general_tier_flag) is set to “1”. In addition, the low bit rate flag(general_lower_bit_rate_constraint_flag) is set to “0” or “1”, and theidentification data (general_higher_bit_rate_indication_flag) is set to“1”. Thus, the parameter HbrFactor becomes “4” or “2”, and the parameterShFactor becomes “4” or “2”.

As a result, the maximum value of the bit rate becomes four times or twotimes of the bit rate corresponding to the main tier, and thecompression rate of the lowest limit becomes ¼ or ½ of the compressionrate of the lowest limit represented by MinCr.

In addition, instead of the parameter HbrFactor′, the parameterHbrFactor may be configured to be used.

In the description presented above, while the compression rate of thelowest limit is adjusted based on the identification data, asillustrated in FIG. 20, by setting MinCr for each tier, the compressionrate of the lowest limit may be adjusted.

Second Embodiment

We nave added our arguments to section 1 and added our proposed systemas a new “Option 2”, adding working draft text to the annex.

We have pulled FormatCapabilityFactor into the maximum bit ratecalculations by making CpbBr [Vcl|Nal]Factor a function of it. If thatis too much change, CpbBr [Vcl|Nal]Factor is profile-specific anyway andwe can just change that value to accommodate 4:4:4 12-bit or 16-bit (Thevalue is used only for determining maximum bit rate).

We have modified Option 1 with a few extra ShFactors, which shouldeffect the desired quadrupling of the bit rate when in super-high tier.Note that Option 1 does not address the inability of the CPE to store awhole frame, but this is not an actual requirement, just a nice-to-have.

If we go with this option, we will need to tweak the values ofCpbBr[Vcl|Nal]Factor for the higher profiles in order to take maximumbit depth into account as well as chroma format.

Variant 1 of Option 1 may have to be removed as it does not address theproblem with maximum bit rate—only MinCR is modified.

Opinion 3 can work as long as it is accepted into O1005_v4 (i.e.alongside the HbrFactor stuff) If this case is used, then, as withOption 1, we will need to tweak CpbBr[Vcl|Nal|Factor for the higherprofiles to get the maximum bit rates we want for higher bit depths.

As an aside, there seems to be a missing edit in subclause E.2.3 in theworking draft. Specifically the definitions of the inferred defaultvalues for cpb_size_value_minus1 and cpb_size_du_value_minus1 stillcontain references to CpbBrVclFactor and CpbBrNalFactor even thoughthese have been fined to 1000 and 1100 respectively when determining CPBsizes.

Abstract

The constraint on MinCR is specified to limit the worst case of CABACdecoding process. In the current level/tier definition, the value ofMinCR is the same for both Main Tier and High Tier. The current MinCRwas defined mainly for Main Tier and larger value is used for higherlevel. For example, MinCR is equal to 8 for level 5.1 (4K@60p). Howeverthis value is too large for high end professional equipments. Inaddition, the implicit minimum sequence compression ratio defined by themaximum bit rate and maximum luma sample rate is asserted to be overlyrestrictive when using all-intra coding or higher bit depths and chromaformats, to the point where MinCr becomes superfluous. This contributionproposes to reduce the value of MinCR and introduce a new tier to allowincreased bit rates for high end professional applications.

1 MinCR

The level and tier limits are specified in FIG. 1 (Table A-1) and FIG. 2(Table A-2).

Main tier is defined mainly for consumer applications, higher tier isdefined for high end consumer and professional applications. The bitrate of high tier is higher than that of main tier. However MinCR is thesame for both main and high tier.

For large picture, e.g. 4K video, higher value of MinCR is used toreduce the worst case complexity of CABAC. For example, the value ofMinCR is 8 for level 5.1 (4K@60p).

However MinCR=8 is too high for high end applications. When 4K@60p videois encoded at 160 Mbps, this MinCR>8 is normally occurred for I pictureand it is not a rare case.

In addition, MinCR=8 is too high when the target quality is visuallylossless. It is hard to achieve visually lossless coding due to thisconstraint.

As well as the per-frame minimum compression ratio defined by MinCR,there also exists an implied minimum compression ratio for the sequenceas a whole defined by the maximum bit rate and maximum intra samplerate. In the example given above, if 4K@60p video is to be coded usinglevel 5.1 (the lowest level at which such video can be coded), thoughthe MinCr value is 8, the actual minimum compression ratio is definedas:

Maximum luma sample rate=534,773,760 samples/second (equal to 4096×2176@60p) Maximum bit rate(high tier)=160,000,000 bits/second (mainprofile CpbBrVclFactor)

Baseband bit rate=534,773,760*1.5(4:2:0)*8 (bits/sample)=6,417,285,120bits/second

Minimum sequence compression ratio=baseband bit rate/maximum bitrate=40:1

Though this degree of compression may be acceptable for inter-codedsequences (as the inter pictures can be compressed to a far greaterdegree than the intra pictures), for all-intra-coded sequences, theMinCR-defined minimum compression ratio becomes purely academic as thesequence must be compressed 40:1 using only intra coding.

Because of this, it will be necessary to modify the maximum bit ratewhen all-intra coding is in use if the value of MinCR is to be at allrelevant. In addition, the above example rises only 4:2:0 8-bit coding.When higher bit depths and more detailed chroma formats are in use, theminimum compression ratio will be even larger. For example, when using4:4:4 12-bit processing, the minimum compress ion ratio doubles to 80:1.

Furthermore, for certain profile and level combinations (e.g. HD@30p4:4:4 12-bit coded using level 4−MinCr=4 (MinCrBase)*0.5(MinCrScaleFactor)=2), the Coded Picture Buffer (CPB) (30,000,000 bitsat level 4) will not be able to contain a whole coded picture if thatpicture has compression ratio equal to MinCR. Though there is no actualrequirement that a decoder must hold a whole picture at once, it mayintroduce difficulties in implementation.

There is, however a requirement that the decoder be able to removepictures from the CPB in real-time. The maximum coded picture size isgiven as (example: HD@30p 4:4:4 12-bit coded in level 4):

Baseband picture size=1920*1080*3*12=74,649,600 bits

Maximum compressed picture size=baseband size/MinCR=37,324,800 bits

If the CPB is filled at the maximum bit rate (45,000,000 bits/second for4:4:4 12-bit at level 4 (high tier)), then the decoder will be receivingonly 1.2 pictures/second and will not be able to decode the sequence inreal-time. It would be possible to alleviate this problem by coding in ahigher level, though even for HD@30p, level 6.1 or higher is required toprovide a high enough maximum bit rate.

Therefore, this contribution proposes to reduce the value of MinCR andintroduce a new tier for high-end professional applications.

2 Proposal

Version 4 of the current working draft text (JCTVC-O1005_v4) allowsMinCR to be halved and maximum bit rate to be doubled via HbrFactor whenall-intra coding is in use. Though this modification to MinCR issufficient, a greater change to the maximum bit rate is needed for usein high-end professional applications.

Since the product development of HEVC had already been started and thereare some deployments in market. Therefore the changes should be made notaffecting such deployments. There could be the following options.

Option 1: Add another tier, e.g. “Super high tier”, and MinCR is reducedfor such tier. A flag is introduced to indicate to reduce MinCR and toincrease Max Bitrate and define profile specific level/tier constraintsfor RExt profiles. For those profiles, MinCR of high tier is interpretedto lower value.

Option 2: odd Super high tier and increase Max Bitrate for this tier.Reduce MinCR using the method specified in JCTVC-O1005_v4.

Option 3: If all current deployments are for main tier and if all peopleagree, it may be possible to change the values of MinCR and Max Bitrateof high tier. It means the changes are applied to Main and Main 10profiles.

The text changes for each options are summarized in the Annex.

If nobody object, option 2 is desirable as it most easily integrateswith the current working draft without requiring modification to version1, but option 1 and option 3 could be fine too. It should be decidedduring the discussion in San Jose meeting.

3 Conclusions

In this contribution, three options to reduce MinCR and increase themaximum bit rate are proposed. One of those options should be adopted tosupport visually lossless coding in HEVC.

4 Annex 4.1 Option 1

Add flag to indicate higher bit rate(general_higher_bit_rate_indication_flag) (FIG. 3)

Change the level limits as follows

Maximum bit rate is derived as follows.

VCL HRD parameter: Max bit rate=CpbBrVclFactor*MaxBP.

NAL HPD parameter: Max bit rate=CpbBrNalFactor*MaxBP.

MinCR is derived as follows

MinCR=Max(1, MinCrBase*MinCrScaleFactor)

where MaxBP and MinCrBase are defined in Table A-2.

CpbBrVclFactor, CpbBrNalFactor and MinCrScaleFactor are defined in TableA-3 as follows.

Variant 1: Reduce MinCR for high tier (FIG. 21 and FIG. 22)

HbrFactor=(2−general_lower_bit_rate_constraint_flag)+2*general_higher_bit_rate_indication_flag

VCL HPD parameter: Max bit rate=CpbBrVclFactor*MaxBR

NAL HPD parameter: Max bit rate=CpbBrNalFactor*MaxBR

MinCR=Max(1, MinCrBase*MinCrScaleFactor)

Variant 2: Add “Super high tier” and MinCR of Main and Main 10 are onlychanged for new tier (FIG. 23 and FIG. 24)

HbrFactor=(2−general_lower_bit_rate_constraint_flag)+2*general_higher_bit_rate_indication_flag

ShFactor=1+(!general_lower_bit_rate_constraint_flag)*general_higher_bit_rate_indication_flag

Variant 3: Add “Super high tier” and modify MinCP for high tier (FIG. 25and FIG. 26)

HbrFactor=2−general_lower_bit_rate_constraint_flag+2*general_higher_bit_rate_indication_flag

ShFactor=1+(!general_lower_bit_rate_constraint_flag)*3*general_higher_bit_rate_indication_flag

VCL HPD parameter: Max bit rate=CpbBrVclFactor*MaxBR

NAL HRD parameter: Max bit rate=CpbBrNalFactor*MaxBR

MinCR=Max(1, MinCrBase*MinCrScaleFactor)

4.2 Option 2

Introduce super high tier. Use method as in JCTVC-O1005_v4 to modifyMinCR.

4.2.1 Profile, tier and level syntax (FIG. 27)

The value of general_super_high_tier_flag shall be 0 except when thevalue of general_tier_flag is 0. The value 1 ofgeneral_super_high_tier_flag when general_tier_flag is 1 is reserved forfuture use.

Change the level limits as follows

Maximum bit rate is derived as follows.

VCL HRD parameter: Max bit rate=CpbBrVclFactor*MaxBP

NAL HRD parameter: Max bit rate=CpbBrNalFactor*MaxBP

MinCR is derived as follows

MinCR=Max(1, MinCrBase*MinCrScaleFactor)

HbrFactor is derived as follows

HbrFactor=2−general_lower_bit_rate_constraint_flag

CpbBrVclFactor and CpbBrNalFactor are derived as follows.

CpbBrVclFactor=((FormatCapabilityFactor*HbrFactor)/1.5)*1000

CpbBrNalFactor=((FormatCapabilityFactor*HbrFactor)/1.5)*1100

where MaxBR and MinCrBase are defined in Table A-2,FormatCapabilityFactor and MinCrScaleFactor are defined in Table A-3.

Table A 1—General tier and level limits (FIG. 28)

Table A 2—Tier and level limits for the Monochrome 12, Monochrome 16,Main, Main 10, Main 12,

Main 4:2:2 10, Main 4:2:2 12, Main 4:4:4, Main 4:4:4 10, and Main 4:4:412, Main Intra, Main 10 Intra, Main 12 Intra, Main 4:2:2 10 Intra, Main4:2:2 12 Intra, Main 4:4:4 Intra, Main 4:4:4 10 Intra, and Main 4:4:4 12Intra profiles (FIG. 29)

Table A 3—Specification of FormatCapabilityFactor and MinCrScaleFactor(FIG. 30 and FIG. 31)

4.3 Option 3

Replace Table A-1 and A-2 with:

Table A 1—General tier and level limits (FIG. 32)

Table A 2—Tier and level limits for the Main and Main 10 profiles (FIG.33)

Third Embodiment Description of Computer According to Present Disclosure

A series of the processes described above can be executed either byhardware or by software. In a case where the series of the processes isexecuted by software, a program configuring the software is installed toa computer. Here, the computer includes a computer that is built indedicated hardware, a computer such as a general-purpose personalcomputer that can execute various functions by installing variousprograms thereto, and the like.

FIG. 34 is a block diagram that illustrates an example of the hardwareconfiguration of a computer that executes the series of processesdescribed above according to a program.

In the computer, a Central Processing Unit (CPU) 201, a Read Only Memory(ROM) 202, and a Random Access Memory (RAM) 203 are interconnected by abus 204.

In addition, an input/output interface 205 is connected to the bus 204.An input unit 206, an output unit 207, a storage unit 208, acommunication unit 209, and a drive 210 are connected to theinput/output interface 205.

The input unit 206 is configured by a keyboard, a mouse, a microphone,and the like. The output unit 207 is configured by a display, a speaker,and the like. The storage unit 208 is configured by a hard disk, anon-volatile memory, and the like. The communication unit 209 isconfigured by a network interface and the like. The drive 210 drives amagnetic disk, an optical disc, a magneto-optical disk, or a removablemedium 211 such as a semiconductor memory.

In the computer configured as above, the CPU 201, for example, loads aprogram stored in the storage unit 208 into the RAM 203 through theinput/output interface 205 and the bus 204 and executes the loadedprogram, thereby executing the series of the processes described above.

The program executed by the computer (CPU 201), for example, may beprovided with being recorded on a removable medium 211 as a packagemedium or the like. In addition, the program can be provided through awired or wireless transmission medium such as a local area network, theInternet, or digital satellite broadcast.

In the computer, by loading the removable medium 211 into the drive 210,a program can be installed to the storage unit 208 through theinput/output interface 205. In addition, the program can be installed,to the storage unit 208 by receiving the program, using thecommunication unit 209 through a wired or wireless transmission medium.Furthermore, the program can be installed to the ROM 202 or the storageunit 208 in advance.

In addition, the program executed by the computer may be a computer thatexecutes the processes in a time series along the sequence described inthis specification or a program that executes the processes in aparallel manner or at necessary timing such as at the timing of beingcalled.

Fourth Embodiment Example of Configuration of Television Apparatus

FIG. 35 illustrates an example of the schematic configuration of atelevision apparatus to which the present technology is applied. Thetelevision apparatus 300 has an antenna 901, a tuner 902, ademultiplexer 903, a decoder 904, a video signal processing unit 905, adisplay unit 906, an audio signal processing unit 907, a speaker 908, anexternal interface unit 909. In addition, the television apparatus 900has a control unit 910, a user interface unit 911, and the like.

The tuner 902 selects a desired channel from broadcast wave signalsreceived by the antenna 901, performs demodulation, and outputs anacquired coded bit stream to the demultiplexer 903.

The demultiplexer 903 extracts packets of a video and an audio of aprogram, that is a viewing target from the coded bit stream and outputsdata of the extracted packets to the decoder 904. In addition, thedemultiplexer 903 supplies the packets of data of an Electronic ProgramGuide (EPG) or the like to the control unit 910. Furthermore, thedemultiplexer may perform descrambling using a demultiplexer or the likein a case where the coded bit stream is scrambled.

The decoder 904 performs a process of decoding the packets and outputsvideo data generated by the decoding process to the video signalprocessing unit 905 and outputs audio data to the audio signalprocessing unit 907.

The video signal processing unit 905 performs noise elimination, videoprocessing corresponding to a user setting, and the like for the videodata. The video signal processing unit 905 generates video data of aprogram to be displayed on the display unit 906, image data acquired bya process based on an application supplied through a network, and thelike. In addition, the video signal processing unit 905 generates videodata, for example, used for displaying a menu screen for an itemselection or the like and superimposes the generated video data onto thevideo data of a program. The video signal processing unit 905 generatesa drive signal based on the video data generated in this way and drivesthe display unit 906.

The display unit 906 drives a display device (for example, a liquidcrystal display or the like) based on the drive signal supplied from thevideo signal processing unit 905, thereby displaying the video of aprogram and the like.

The audio signal processing unit 907 performs a predetermined processsuch as noise removal for the audio data, performs a D/A conversionprocess of audio data after the process or an amplification processthereof, and supplies resultant data to the speaker 908, therebyperforming audio output.

The external interface unit 909 is an interface used for a connection toan external device or a network and transmits/receives data such asvideo data or audio data.

The user interface unit 911 is connected to the control unit 910. Theuser interface unit 911 is configured by an operation switch, a remotecontrol signal reception unit, and the like and supplies an operationsignal according to a user operation to the control unit 910.

The control unit 910 is configured by a Central Processing Unit (CPU), amemory, and the like. The memory stores a program executed by the CPU,various kinds of data that is necessary for the process performed by theCPU, EPG data, data acquired through a network, and the like. Theprogram that is stored in the memory is read and executed by the CPU atpredetermined timing such as start-up of the television apparatus 900.By executing the program, the CPU performs control of each unit suchthat the television apparatus 900 operates in accordance with a useroperation.

In addition, in the television apparatus 900, in order to connect thetuner 902, the demultiplexer 903, the video signal processing unit 905,the audio signal processing unit 907, the external interface unit 909,and the like to the control unit 910, a bus 912 is disposed.

In the television apparatus configured in this way, the function of thedecoding apparatus (decoding method) according to the presentapplication is implemented in the decoder 904. Accordingly, a codedstream in which restrictions of the compression rate of the lowest limitand the maximum value of the bit rate are adjusted can be decoded.

Fifth Embodiment Configuration Example of Mobile Phone

FIG. 36 illustrates the schematic configuration of a mobile phone towhich the present technology is applied. The mobile phone 920 includes:a communication unit 922; an audio codec 923; a camera unit 926; animage processing unit 927; a multiplexing/separating unit 928; arecording/reproducing unit 929; a display unit 930; and a control unit931. These are interconnected through the bus 933.

In addition, the antenna 921 is connected to the communication unit 922,and the speaker 924 and the microphone 925 are connected to the audiocodec 923. Furthermore, the operation unit 932 is connected to thecontrol unit 931.

The mobile phone 920 performs various operations such as transmissionand reception of an audio signal, transmission and reception of anelectronic mail and image data, image capturing, and data recording invarious modes such as a voice call mode and a data communication mode.

In the voice call mode, an audio signal generated by the microphone 925is converted into audio data or compressed by the audio codec 923, and aresultant signal is supplied to the communication unit 922. Thecommunication unit 922 performs a modulation process, a frequencyconversion process, and the like for the audio data, thereby generatinga transmission signal. In addition, the communication unit 922 suppliesa transmission signal to the antenna 921 so as to be transmitted to abase station not illustrated in the figure. Furthermore, thecommunication unit 922 performs an amplification process, a frequencyconversion process, a demodulation process, and the like for a receptionsignal received by the antenna 921 and supplies acquired audio data tothe audio codec 923. The audio codec 923 performs data decompression ofthe audio data and converts the audio data into an analog audio signaland outputs a resultant signal to the speaker 924.

In addition, in the data communication mode, in a case where a mail istransmitted, the control unit 931 receives character data input by anoperation for the operation unit 932 and displays the input characterson the display unit 930. Furthermore, the control unit 931 generatesmail data based on a user's instruction from the operation unit 932 andsupplies the generated mail data to the communication unit 922. Thecommunication unit 922 performs a modulation process, a frequencyconversion process, and the like for the mail data and transmits anacquired transmission signal from the antenna 921. In addition, thecommunication unit 922 performs an amplification process, a frequencyconversion process, a demodulation process, and the like for thereception signal received by the antenna 921, thereby restoring the maildata. This mail data is supplied to the display unit 930, whereby thecontent of the mail is displayed.

In addition, the mobile phone 920 can store the received mail data in astorage medium using the recording/reproducing unit 929. The storagemedium may be an arbitrary rewritable storage medium. For example, thestorage medium is a semiconductor memory such as a RAM or a built-intype flash memory, a hard disk, a magnetic disk, a magneto-optical disk,an optical disc, or a removable medium such as a Universal Serial Bus(USB) memory or a memory card.

In the data communication mode, in a case where image data istransmitted, the image data generated by the camera unit 926 is suppliedto the image processing unit 927. The image processing unit 927 performsa coding process of the image data, thereby generating coded data.

The multiplexing/separating unit 928 multiplexes coded data generated bythe image processing unit 927 and audio data supplied from the audiocodec 923 in accordance with a predetermined system and suppliesmultiplexed data to the communication unit 922. The communication unit922 performs a modulation process, a frequency conversion process, andthe like of the multiplexed data and transmits an acquired transmissionsignal from the antenna 921. In addition, the communication unit 922performs an amplification process, a frequency conversion process, ademodulation process, and the like for the reception signal received bythe antenna 921, thereby restoring the multiplexed data. Thismultiplexed data is supplied to the multiplexing/separating unit 928.The multiplexing/separating unit 928 separates the multiplexed data andsupplies coded data to the image processing unit 927 and supplies audiodata to the audio codec 923. The image processing unit 927 performs adecoding process of the coded data, thereby generating image data. Thisimage data is supplied to the display unit 930, whereby the receivedimage is displayed. The audio codec 923 converts audio data into ananalog audio signal and supplies the converted analog audio signal tothe speaker 924, thereby outputting the received audio.

In the mobile phone device configured in this way, the functions of thecoding apparatus and the decoding apparatus (a coding method and adecoding method) accord ing to the present application are implementedin the image processing unit 927. For this reason, restrictions of thecompression rate of the lowest limit and the maximum value of thebitrate can be adjusted. In addition, a coded stream of which thecompression rate of the lowest limit and the maximum value of the bitrate are adjusted can be decoded.

Sixth Embodiment Configuration Example of Recording/ReproducingApparatus

FIG. 37 illustrates the schematic configuration of arecording/reproducing apparatus to which the present technology isapplied. The recording/reproducing apparatus 940, for example, recordsaudio data and video data of a received broadcast program on a recordingmedium and provides the recorded data for a user at timing according toa user's instruction. In addition, the recording/reproducing apparatus940, for example, may acquire audio data and video data from anotherdevice and record the audio data and the video data on a recordingmedium. Furthermore, the recording/reproducing apparatus 940 decodes andoutputs the audio data and the video data, which are recorded on therecording medium, whereby the display of an image or the output of anaudio can be performed in a monitor device or the like.

The recording/reproducing apparatus 940 has a tuner 941, an externalinterface unit 942, an encoder 943, a Hard Disk Drive (HDD) unit 944, adisk drive 945, a selector 946, a decoder 947, an On-Screen Display(OSD) unit 948, a control unit 949, and a user interface unit 950.

The tuner 941 selects a desired channel from among broadcast signalsreceived by an antenna not illustrated in the figure. The tuner 941outputs a coded bit stream acquired by demodulating a reception signalof the desired channel to the selector 946.

The external interface unit 942 is configured by at least one of anIEEE1394 interface, a network interface unit, a USB interface, a flashmemory interface, and the like. The external interface unit 942 is aninterface for a connection to an external device, a network, a memorycard, or the like and performs data reception of video data, audio data,and the like to be recorded.

When the video data and the audio data supplied from the externalinterface unit 942 are not coded, the encoder 943 codes the video dataand the audio data in accordance with a predetermined system and outputsa coded bit stream to the selector 946.

The HDD unit 944 records content data such as videos and audios, variousprograms, other data, and the like on a built-in hard disk and reads therecorded data from the hard disk at the time of reproduction or thelike.

The disk drive 945 performs signal recording and signal reproducing fora loaded optical disc. The optical disc, for example, is a DVD disc (aDVD-Video, a DVD-RAM, a DVD−R, a DVD−RW, a DVD+R, a DVD+RW, or thelike), a Blu-ray (registered trademark) disc, or the like.

When a video or an audio is recorded, the selector 946 selects a codedbit stream supplied from the tuner 941 or the encoder 943 and suppliesthe selected code bit stream to one of the HDD unit 944 and the diskdrive 945. In addition, when a video or an audio is reproduced, theselector 946 supplies a coded bit stream output from the HDD unit 944 orthe disk drive 945 to the decoder 947.

The decoder 947 performs a decoding process of the coded bit stream. Thedecoder 947 supplies video data that is generated by performing thedecoding process to the OSD unit 948. In addition, the decoder 947outputs audio data that is generated by performing the decoding process.

The OSD unit 948 generates video data used for displaying a menu screensuch as an item selection menu or the like and outputs the generatedvideo data so as to overlap the video data output from the decoder 947.

The user interface unit 950 is connected to the control unit 949. Theuser interface unit 950 is configured by an operation switch, a remotecontrol signal reception unit, and the like and supplies an operationsignal according to a user operation to the control unit 949.

The control unit 949 is configured by using a CPU, a memory, and thelike. The memory stores programs that are executed by the CPU andvarious kinds of data that is necessary for the process performed by theCPU. A program stored in the memory is read and executed by the CPU atpredetermined timing such as the start-up of the recording/reproducingapparatus 940. The CPU executes programs, thereby performing control ofeach unit such that the recording/reproducing apparatus 940 operates inaccordance with a user operation.

In the recording/reproducing apparatus configured in this way, thefunction of the decoding apparatus (decoding method) according to thepresent application is implemented in the decoder 947. For this reason,a coded stream in which restrictions of the compression rate of thelowest limit and the maximum value of the bit rate are adjusted can bedecoded.

Seventh Embodiment Configuration Example of Imaging Apparatus

FIG. 38 is a diagram that illustrates an example of the schematicconfiguration of an imaging apparatus to which the present technology isapplied. The imaging apparatus 960 images a subject and displays theimage of the subject on a display unit or records the image of thesubject on a recording medium as image data.

The imaging apparatus 960 includes: an optical block 961; an imagingunit 962; a camera signal processing unit 963; an image data processingunit 964; a display unit 965; an external interface unit 966; a memoryunit 967; a media drive 968; an OSD unit 969; and a control unit 970. Inaddition, a user interface unit 971 is connected to the control unit970. Furthermore, the image data processing unit 964, the externalinterface unit 966, the memory unit 967, the media drive 968, the OSDunit 969, the control unit 970, and the like are interconnected througha bus 972.

The optical block 961 is configured by using a focusing lens, adiaphragm mechanism, and the like. The optical block 961 forms theoptical image of a subject on the imaging surface of the imaging unit962. The imaging unit 962 is configured by using a CCD or CMOS imagesensor and generates an electrical signal according to the optical imagethrough a photoelectric conversion and supplies the generated electricalsignal to the camera signal processing unit 963.

The camera signal processing unit 963 performs various kinds of camerasignal processing such as a knee correction, a gamma correction, and acolor correction for the electrical signal supplied from the imagingunit 962. The camera signal processing unit 963 supplies image dataafter the camera signal processing to the image data processing unit964.

The image data processing unit 964 performs a coding process of theimage data supplied from the camera signal processing unit 963. Theimage data processing unit 964 supplies coded data that is generated byperforming the coding process to the external interface unit 966 or themedia drive 968. In addition, the image data processing unit 964performs a decoding process of the coded data supplied from the externalinterface unit 966 or the media drive 968. The image data processingunit 964 supplies the image data generated by performing the decodingprocess to the display unit 965. In addition, the image data processingunit 964 performs the process of supplying the image data supplied fromthe camera signal processing unit 963 to the display unit 965 andsupplies display data acquired from the OSD unit 969 to the display unit965 with being overlapped with the image data.

The OSD unit 969 generates display data such as a menu screen or an iconthat is configured by symbols, characters, or graphics and outputs thegenerated display data to the image data processing unit 964.

The external interface unit 966, for example, is configured by a USBinput/output terminal and the like and is connected to the printer in acase where an image is printed. In addition, to the external interfaceunit 966, a drive is connected as is necessary, a removable medium suchas a magnetic disk or an optical disc is appropriately installed, and acomputer program read therefrom is installed as is necessary.Furthermore, the external interface unit 966 includes a networkinterface that is connected to a predetermined network such as a LAN orthe Internet. For example, in accordance with an instruction from theuser interface unit 971, the control unit 970 can read coded data fromthe media drive 968 and supply the read coded data from the externalinterface unit 966 to another device connected through a network. Inaddition, the control unit 970 can acquire coded data or image data,which is supplied from another device through a network, through theexternal interface unit 966 and supply the acquired data to the imagedata processing unit 964.

As the recording media driven by the media drive 968, for example, anarbitrary readable/writable removable medium such as a magnetic disk, amagneto-optical disk, an optical disc, or a semiconductor memory isused. In addition, the type of the recording medium as a removablemedium is an arbitrary and thus, may be a tape device, a disk, or amemory card. Furthermore, a non-contact Integrated Circuit (IC) card orthe like may be used as the recording medium.

In addition, by integrating the media drive 968 and the recording mediumtogether, for example, the recording medium may be configured by anon-portable recording medium such as a built-in type hard disk drive oran Solid State Drive (SSD).

The control unit 970 is configured by using a CPU. The memory unit 967stores programs that are executed by the control unit 970, various kindsof data that is necessary for the process performed by the control unit970, and the like. A program stored in the memory unit 967 is read andexecuted by the control unit 970 at predetermined timing such as thestart-up of the imaging apparatus 960. The control unit 970 executesprograms, thereby performing control of each unit such that the imagingapparatus 960 operates in accordance with a user operation.

In the imaging apparatus configured in this way, the functions of thecoding apparatus and the decoding apparatus (a coding method and adecoding method) according to the present application is implemented inthe image data processing unit 964. For this reason, restrictions of thecompression rate of the lowest limit and the maximum value of the bitrate can be adjusted. In addition, a coded stream of which thecompression rate of the lowest limit and the maximum value of the bitrate are adjusted can be decoded.

Eighth Embodiment Other Examples

While the examples of the apparatus, the system, and the like to whichthe present technology is applied have been described as above, thepresent technology is not limited thereto. Thus, the present technologymay be also implemented as all the components mounted, in the apparatusor an apparatus configuring the system such as a processor as a systemLarge Scale Integration (LSI) or the like, a module using a plurality ofprocessors and the like, a unit using a plurality of modules and thelike, and a set acquired by adding another function to the unit (inother words, a part of the configuration of the apparatus).

(Configuration Example of Video Set)

An example of a case where the present technology is implemented as aset will be described, with reference to FIG. 39. FIG. 39 illustrates anexample of the schematic configuration of a video set to which thepresent technology is applied.

Recently, implementation of multiple functions in electronic apparatuseshas been progressed, and, in the development or the manufacture of eachelectronic apparatus, in a case where a part of the configuration isexecuted in sales, provision, or the like, frequently, there is not onlya case where the part is executed as a configuration having one functionbut also a case where the part is executed as one set having multiplefunctions by combining a plurality of configurations having relatedfunctions.

The video set 1300 illustrated in FIG. 39 has such a configuration thathas multiple functions and is acquired by combining a device having afunction relating to coding/decoding (one of coding and decoding or boththereof) of an image with devices having other functions relating to thefunction.

As illustrated in FIG. 39, the video set 1300 includes: a module groupsuch as a video module 1311, an external memory 1312, a power managementmodule 1313, and a front end module 1314 and devices having relatedfunctions such as connectivity 1321, a camera 1322, and a sensor 1323.

A module is a component that has functions having coherence acquired bygathering several component functions relating to each other. A specificphysical configuration is arbitrary, and, for example, a configurationmay be considered in which a plurality of processors having respectivefunctions, electronic circuit components such as a resistor and acapacitor, and other devices are rearranged to be integrated in a wiringboard or the like. In addition, it may be considered to form a newmodule by combining a module with other modules, a processor, or thelike.

In the case of the example illustrated in FIG. 33, the video module 1311is constructed by combining configurations having functions relating toimage processing and includes: an application processor, a videoprocessor, a broadband modem 1333, and an RF module 1334.

A processor is formed by integrating a configuration having apredetermined function on a semi conductor chip through a System On aChip (SoC) and, for example, there is a processor called a Large ScaleIntegration (LSI) or the like. The configuration having a predeterminedfunction may be a logical circuit (hardware configuration), aconfiguration including a CPU, a ROM, a RAM, and the like and a program(software configuration) executed using the components, or aconfiguration acquired by combining both. For example, if may beconfigured such that a processor includes logic circuits, a CPU, a ROM,a RAM, and the like, some functions thereof are realized by logiccircuits (hardware configuration), and the other functions are realizedby a program (software configuration) executed by the CPU.

An application processor 1331 illustrated in FIG. 39 is a processor thatexecutes an application relating to image processing. In order torealize a predetermined function, the application executed by thisapplication processor 1331 not only executes a calculation process butalso may control configurations of the inside/outside of the videomodule 1311 such as a video processor 1332 and the like as is necessary.

A video processor 1332 is a processor that has a function relating tocoding/decoding (one of coding and decoding or both coding and decoding)of an image.

The broadband modem 1333 is a processor (or a module) relating to wiredor wireless (or wired and wireless) broadband communication performedthrough a broadband line such as the Internet or a public telephonenetwork. For example, the broadband modem 1333 converts data (digitalsignal) to be transmitted into an analog signal through digitalmodulation or the like or demodulates a received analog signal so as tobe converted into data (digital signal). For example, the broadbandmodem 1333 can perform digital modulation/demodulation of arbitraryinformation such as image data processed by the video processor 1332, astream in which the image data is coded, an application program, settingdata, and the like.

The RF module 1334 is a module that performs frequency conversion,modulation/demodulation, amplification, a filter process, and the likefor a Radio Frequency (RF) signal that is transmitted or receivedthrough an antenna. For example, the RF module 1331 performs thefrequency conversion and the like for a dedicated line connection systemsignal generated by the broadband modem 1333, thereby generating an RFsignal. In addition, for example, the RF module 1334 performs thefrequency conversion and the like for an RF signal received through thefront end module 1314, thereby generating a dedicated line connectionsystem signal.

As denoted by a dotted line 1341 in FIG. 33, the application processor1331 and the video processor 1332 may be integrated so as to beconfigured as one processor.

The external memory 1312 is a module that is arranged outside the videomodule 1311 and has a memory device used by the video module 1311. Whilethe memory device of the external memory 1312 may be realized by acertain physical configuration, generally, the memory device isfrequently used for storing data of a large volume such as image dataconfigured in units of frames. Accordingly, it is preferable that memorydevice is realized by a semiconductor memory of a large capacity such asa dynamic random Access Memory (DRAM) at a relatively low cost.

The power management module 1313 manages and controls supply of power tothe video module 1311 (each configuration arranged inside the videomodule 1311).

Tire front end module 1314 is a module that provides a front endfunction (a circuit at the transmission/reception end on the antennaside) for the RF module 1334. As illustrated in FIG. 39, the front endmodule 1314, for example, includes an antenna unit 1331, a filter 1352,and an amplification unit 1353.

The antenna unit 1351 includes an antenna that transmits and receiveswireless signals and peripheral configurations. The antenna unit 1351transmits a signal supplied from the amplification unit 1353 as awireless signal and supplies the received wireless signal to the filter1352 as an electrical signal (RF signal). The filter 1352 performs afilter process and the like for the RF signal received through theantenna unit 1351 and supplies the RF signal after the process to the RFmodule 1334. The amplification unit 1353 amplifies the RF signalsupplied from the RF module 1334 and supplies the amplified RF signal tothe antenna unit 1351.

The connectivity 1321 is a module that has a function relating to aconnection with the outside. The physical configuration of theconnectivity 1321 is arbitrary. For example, the connectivity 1321includes a configuration having a communication function according to acommunication standard other than a communication standard with whichthe broadband modem 1333 compliant, an external in put/output terminal,and the like.

For example, the connectivity 1321 may be configured to include a modulethat has a communication function compliant with a radio communicationstandard such as Bluetooth (registered trademark), IEEE 802.11 (forexample, Wireless Fidelity; registered trademark (Wi-Fi)), Bear FieldCommunication (NFC), or InfraRed Data Association (IrDA), an antennathat transmits and receives signals compliant with the standard, and thelike. In addition, for example, the connectivity 1321 may be configuredto include a module that has a communication function compliant with awired communication standard such as Universal Serial Bus (USB) orHigh-Definition Multimedia Interface (HDMI (registered trademark)) andterminals compliant with the standard. Furthermore, for example, theconnectivity 1321 may be configured to have another data (signal)transmission function of an analog input/output terminal or the like.

In addition, the connectivity 1321 may be configured to include a deviceof the transmission destination of data (signal). For example, theconnectivity 1321 may be configured to include a drive (including notonly a drive of a removable medium but also a hard disk, a Solid StateDrive (SSD), a Network Attached Storage (NAS), and the like) thatreads/writes data from/into a recording medium such as a magnetic disk,an optical disc, a magneto-optical disk, or a semi conductor memory. Inaddition, the connectivity 1321 may be configured to include an outputdevice (a monitor, a speaker, or the like) of an image or a voice.

The camera 1322 is a module that has a function for acquiring image dataof an object by imaging the object. The image data acquired by theimaging process performed by the camera 1322, for example, is suppliedto the video processor 1332 and is coded.

The sensor 1323 is a module that has the function of an arbitrary sensorsuch a sound sensor, an ultrasonic sensor, an optical sensor, anilluminance sensor, an infrared sensor, an image sensor, a rotationsensor, an angle sensor, an angular velocity sensor, a velocity sensor,an acceleration sensor, a tilt sensor, a magnetic identification sensor,an impact sensor, or a temperature sensor. Data detected by the sensor1323, for example, is supplied to the application processor 1331 and isused by the application and the like.

The configuration described above as the module may be realized as theprocessor. To the contrary, the configuration described above as theprocessor may be realized as the module.

In the video set 1300 having the above-described, configuration, as willbe described later, the present technology may be applied to the videoprocessor 1332. Accordingly, the video set 1300 may be executed as a setto which the present technology is applied.

(Configuration Example of Video Processor)

FIG. 40 illustrates an example of the schematic configuration of thevideo processor 1332 (FIG. 39) to which the present technology isapplied.

In the case of the example illustrated in FIG. 40, the video processor1332 has a function for receiving inputs of a video signal and art audiosignal and coding the video signal and the audio signal according to apredetermined system and a function for decoding coded video data andcoded audio data and reproducing and outputting a video signal and audiosignal.

As illustrated in FIG. 40, the video processor 1332 includes: a videoinput processing unit 1401; a first image enlargement/reduction unit1402; a second image enlargement/reduction unit 1403; a video outputprocessing unit 1404; a frame memory 1405; and a memory control unit1406. In addition, the video processor 1332 includes: anencoding/decoding engine 1407; video Elementary Stream (ES) buffers1408A and 1408B; and audio ES buffers 1409A and 1409B. Furthermore, thevideo processor 1332 includes: an audio encoder 1410; an audio decoder1411; a multiplexing unit (Multiplexer (MUX)) 1412; a demultiplexingunit (Demultiplexer (DMUX)) 1413; and a stream buffer 1414.

The video input processing unit 1401, for example, acquires a videosignal input from the connectivity 1321 (FIG. 39) or the like andconverts the video signal into digital image data. The first imageenlargement/reduction unit 1402 performs a format conversion, an imageenlargement/reduction process, and the like for the image data. Thesecond image enlargement/reduction unit 1403 performs an imageenlargement/reduction process according to a format of the outputdestination through the video output processing unit 1404, similarformat conversion as that of the first image enlargement/reduction unit1402, an image enlargement/reduction process, and the like for the imagedata. The video output processing unit 1404 performs a formatconversion, a conversion into an analog signal, and the like for theimage data and outputs a resultant signal, for example, to theconnectivity 1321 (FIG. 39) or the like as a reproduced video signal.

The frame memory 1405 is a memory for image data that is shared by thevideo input processing unit 1401, the first image enlargement/reductionunit 1402, the second image enlargement/induction unit 1403, the videooutput processing unit 1404, and the encoding/decoding engine 1407. Theframe memory 1405, for example, is realized by a semiconductor memorysuch as a DRAM.

The memory control unit 1406 receives a synchronization signal from theencoding/decoding engine 1407 and controls accesses to the frame memory1405 for wiring/reading according to an access schedule for accessingthe frame memory 1405 that is written in an access management table1406A. The access management table 1406A is updated by the memorycontrol unit 1406 in accordance with the processes executed by theencoding/decoding engine 1407, the first image enlargement/reductionunit 1402, the second image enlargement/reduction unit 1403, and thelike.

The encoding/decoding engine 1407 performs an encoding process of imagedata and a decoding process of a video stream that is data acquired bycoding the image data. For example, the encoding/decoding engine 1407codes the image data read from the frame memory 1405 and sequentiallywrites the image data into the video ES buffer 1408A as a video stream.In addition, for example, the encoding/decoding engine 1407 sequentiallyreads and decodes video streams supplied from the video ES but far 1408Band sequentially writes the decoded video streams into the frame memory1405 as image data. The encoding/decoding engine 1407 uses the framememory 1405 as a work area in such coding and decoding processes. Inaddition, the encoding/decoding engine 1407 outputs a synchronisationsignal to the memory control unit 1406, for example, at timing when aprocess for each macro block is started.

The video ES buffer 1408A buffers a video stream generated by theencoding/decoding engine 1407 and supplies the buffered video stream tothe multiplexing unit (MUX) 1412. The video ES buffer 1408B buffers avideo stream supplied from the demultiplexing unit (DMUX) 1413 andsupplies the buffered video stream to the encoding/decoding engine 1407.

The audio ES buffer 1409A buffers an audio stream generated by the audioencoder 1410 and supplies the buffered audio stream to the multiplexingunit (MUX) 1412. The audio ES buffer 1409B buffers an audio streamsupplied from the demultiplexer (DMUX) 1413 and supplies the bufferedaudio stream to the audio decoder 1411.

The audio encoder 1410, for example, converts an audio signal, forexample, input from the connectivity 1321 (FIG. 39) or the like into adigital signal and codes the converted digital signal according to apredetermined system such as an MPEG audio system or an Audio Codenumber 3 (AC3) system. The audio encoder 1410 sequentially writes audioscreams each being data acquired by coding an audio signal into theaudio ES buffer 1409A. The audio decoder 1411 decodes the audio streamsupplied from the audio ES buffer 1409B and, for example, performs aconversion into an analog signal, and the like for the decoded audiostream and supplies a resultant signal, for example, to the connectivity1321 (FIG. 39) or the like as a reproduced audio signal.

The multiplexing unit (MUX) 1412 multiplexes a video stream and an audiostream. A method of this multiplexing process (in other words, theformat of a bit stream generated by the multiplexing process) isarbitrary. In addition, in the multiplexing process, the multiplexingunit (MUX) 1412 may add predetermined header information and the like tothe bit stream. In other words, the multiplexing unit (MUX) 1412 canconvert the format of a scream through the multiplexing process. Forexample, by multiplexing a video stream and an audio stream, themultiplexing unit (MUX) 1412 converts the streams into a transportstream that is a bit stream of a transmission format. In addition, forexample, by multiplexing the video stream and the audio stream, themultiplexing unit (MUX) 1412 converts the streams into data (file data)of a recording file format.

The demultiplexing unit (DMUX) 1413 demultiplexes a bit stream in whicha video stream and an audio stream are multiplexed using a methodcorresponding to the multiplexing process performed by the multiplexingunit (MUX) 1412. In other words, the demultiplexing unit (DMUX) 1413extracts a video stream and an audio stream from the bit stream readfrom the stream buffer 1414 (separates the video stream and the audiostream from each other). In other words, the demultiplexing unit (DMUX)1413 can convert the format of a stream through the demultiplexingprocess (an inverse conversion of the conversion performed by themultiplexing unit (MUX) 1412). For example, the demultiplexing unit(DMUX) 1413 acquires a transport stream supplied, for example, from theconnectivity 1321, the broadband modem 1333, or the like (all FIG. 39)through the stream buffer 1414 and demultiplexes the supplied transportstream, thereby converting the transport stream into a video stream andan audio stream. In addition, for example, the demultiplexing unit(DMUX) 1413 acquires file data, for example, read from various recordingmedia by the connectivity 1321 (FIG. 39) through the stream buffer 1414and demultiplexes the acquired file data, thereby converting the filedata into a video stream and an audio stream.

The stream buffer 1414 buffers the bit stream. For example, the streambuffer 1414 buffers the transport stream supplied from the multiplexingunit (MUX) 1412 and supplies the buffered transport stream, for example,to the connectivity 1321, the broadband modem 1333 (all FIG. 39), or thelike at predetermined tinting or based on a request from the external,or the like.

In addition, for example, the stream buffer 1414 buffers the file datasupplied from the multiplexing unit (MUX) 1412 and supplies the bufferedfile data, for example, to the connectivity 1321 (FIG. 39) or the likeat predetermined timing, a request from the external, or the like so asto be recorded on various recording media.

Furthermore, the stream buffer 1414 buffers a transport stream, forexample, acquired through the connectivity 1321, the broadband modem1333, or the like (all FIG. 39) and supplies the buffered transportstream to the demultiplexing an it (DMUX) 1413 at predetermined timingor cased on a request from the external or the like.

In addition, the stream buffer 1414 buffers file data read from variousrecording media by the connectivity 1321 (FIG. 39) or the like andsupplies the buffered file data to the demultiplexing unit (DMUX) 1413at predetermined timing or a request from the external or the like.

Next, an example of the operation of the video processor 1332 havingsuch a configuration will be described. For example, a video signalinput from the connectivity 1321 (FIG. 39) or the like to the videoprocessor 1332 is converted into digital image data of a predeterminedsystem such as a 4:2:2 Y/Cb/Cr system in the video input processing unit1401 and is sequentially written into the frame memory 1403. Thisdigital image data is read by the first image enlargement/reduction unit1402 or the second image enlargement/reduction unit 1403, a formatconversion into a predetermined system such as a 4:2:0 Y/Cb/Cr systemand an enlargement/reduction process are performed for the read digitalimage data, and resultant digital image data is rewritten into the framememory 1405. This image data is coded by the encoding/decoding engine1407 and is written into the video ES buffer 1408A as a video stream.

In addition, an audio signal input from the connectivity 1321 (FIG. 39)or she like to the video processor 1332 is coded by the audio encoder1410 and is written into the audio ES buffer 1409A as an audio stream.

A video stream in tire video ES buffer 1408A and an audio stream in theaudio ES buffer 1409A are read and multiplexed by the multiplexing unit(MUX) 1412 and is converted into a transport stream, file data, or thelike. The transport stream generated by the multiplexing unit (MUX) 1412is buffered in the stream buffer 1414 and then, is output to an externalnetwork, for example, through the connectivity 1321, the broadband modem1333 (all FIG. 39), or the like. In addition, the file data generated bythe multiplexing unit (MUX) 1412 is buffered in the stream buffer 1414and then, is output, for example, to the connectivity 1321 (FIG. 39) orthe like and is recorded on various recording media.

In addition, the transport stream input to the video processor 1332 froman external network, for example, through the connectivity 1321, thebroadband modem 1333 (all FIG. 39), or the like is buffered in thestream buffer 1414 and then, is demultiplexed, by the demultiplexingunit (DMUX) 1413. In addition, the file data that is read from variousrecording media and is input to the video processor 1332, for example,by the connectivity 1321 (FIG. 39) or the like is buffered in the streambuffer 1414 and then, is demultiplexed by the demultiplexing unit (DMUX)1413. In other words, the transport stream or the file data input to thevideo processor 1332 is separated into a video stream and an audiostream by the demultiplexing unit (DMUX) 1413.

The audio stream is supplied to the audio decoder 1411 through the audioES buffer 1409B and is decoded, and an audio signal is reproduced. Inaddition, the video stream is written into the video ES buffer 1408B andthen is sequentially read and decoded by the encoding/decoding engine1407 and is written into the frame memory 1405. The decoded image datais processed to be enlarged or reduced by the second imageenlargement/reduction unit 1403 and is written into the frame memory1405. Then, the decoded image data is read by the video outputprocessing unit 1404, is converted into another format according to apredetermined system such as the 4:2:2 Y/Cb/Cr system or the like, andis further converted into an analog signal, and a video signal isreproduced and output.

In a case where the present technology is applied to the video processor1332 configured as such, the present technology relating to eachembodiment described above may be applied to the encoding/decodingengine 1407. In other words, for example, the encoding/decoding engine1407 may be configured to have the functions of the coding apparatus andthe decoding apparatus according to the first and second embodiments. Byconfiguring as such, the video processor 1332 can have the sameadvantages as those described above with reference to FIGS. 1 to 20.

In addition, in the eroding/decoding engine 1407, the present technology(in other words, the functions of the image coding apparatus and theimage decoding apparatus according to each embodiment described above)may be realized by hardware such as logic circuits or software such asan embedded program or may be realized by both the hardware and thesoftware.

(Another Configuration Example of Video Processor)

FIG. 41 illustrates another example of the schematic configuration of avideo processor 1332 (FIG. 39) to which the present technology isapplied. In the case of the example illustrated in FIG. 41, the videoprocessor 1332 has a function for coding and decoding video dataaccording to a predetermined system.

More, specifically, as illustrated in FIG. 41, the video processor 1332includes: a control unit 1511; a display interface 1512; a displayengine 1513; an image processing engine 1514; and an internal memory1515. In addition, the video processor 1332 includes: a codec engine1516; a memory interface 1517; a multiplexing/demultiplexing unit (MUXDMUX) 1518; a network interface 1519; and a video interface 1520.

The control unit 1511 controls the operations of each processing unitarranged inside the video processor 1332 such as the display interface1512, the display engine 1513, the image processing engine 1514, and thecodec engine 1516.

As illustrated in FIG. 41, the control unit 1511, for example, includesa main CPU 1531, a sub CPU 1532, and a system controller 1533. The mainCPU 1531 executes a program used for controlling the operation of eachprocessing unit arranged, inside the video processor 1332 and the like.The main CPU 1531 generates a control signal according no the program orthe like and supplies the control signal to each processing unit (inother words, controls the operation of each processing unit). The subCPU 1532 achieves an auxiliary role of the main CPU 1531. For example,the sub CPU 1532 executes a child process, a subroutine, or the like ofa program executed by the main CPU 1531. The system controller 1533controls the operations of the main CPU 1531 and the sub CPU 1532 byperforming designation of programs to be executed by the main CPU 1531and the sub CPU 1532 and the like.

The display interface 1512 outputs image data, for example, to theconnectivity 1321 (FIG. 39) and the like under the control of thecontrol unit 1511. For example, the display interface 1512 convertsimage data that is digital data into an analog signal and outputs theanalog signal as a reproduced video signal or the image data that is thedigital data to a monitor device of the connectivity 1321 (FIG. 39) orthe like.

The display engine 1513, under the control of the control unit 1511,performs various conversion processes such as a format conversion, asite conversion, and a color gamut conversion for the image data so asto match the hardware specification of a monitor device displaying theimage and the like.

The image processing engine 1514 performs predetermined image processingsuch as a filter process used for improving the image quality and thelike for the image data under the control of the control unit 1511.

The internal memory 1515 is a memory that is snared by the displayengine 1513, the image processing engine 1514, and the codec engine 1516and is disposed inside the video processor 1332. The internal memory1515, for example, is used for data transfer among the display engine1513, the image processing engine 1514, and the codec engine 1516. Forexample, the internal memory 1515 stores data supplied from the displayengine 1513, the image processing engine 1514, or the codec engine 1516and supplies the data to the display engine 1513, the image processingengine 1514, or the codec engine 1516 as is necessary (for example,according to a request). This internal memory 1515 may be realized byusing any kind of memory device. However, generally, the internal memoryis frequently used for storing data having a small volume such as imagedata in units of blocks and parameters, and accordingly, it ispreferable to realize the internal memory using a semiconductor memoryhaving a relatively small capacity (compared to the external memory1312) and having high response speed such as a Static random AccessMemory (SRAM).

The codec engine 1516 performs processes relating to coding and decodingof image data. The coding/decoding system with which the codec engine1516 is compliant is arbitrary, and the number of coding/decodingsystems may be one or plural. For example, it may be configured suchthat the codec engine 1516 may have a codec function for a plurality ofcoding/decoding systems and be configured to perform coding of imagedata or decoding of coded data by using selected one of thecoding/decoding systems.

In the example illustrated in FIG. 41, the codec engine 1516, forexample, includes MPEG-2 Video 1541, AVC/H.264 1542, HEVC/H.265 1543,HEVC/E.265 (Scalable) 1544, HEVC/H.265 (Multi-view) 1545, and MPEG-DASH1551 as functional blocks for the process relating to the codec.

The MPEG-2 Video 1541 is a functional block that codes or decodes imagedata according to the MPEG-2 system. The AVC/H.264 1542 is a functionalblock that codes or decodes image data according to the AVC system. TheHEVC/H.265 1543 is a functional block that codes or decodes image dataaccording to the HEVC system. The HEVC/H.265 (Scalable) 1544 is afunctional block that performs scalable coding or scalable decoding ofimage data according to the HEVC system. The HEVC/H. 265 (Multi-view)1545 is a functional block that performs multi-view coding or multi-viewdecoding of image data according to the HEVC system.

The MPEG-DASH 1551 is a functional block that transmits and receivesimage data according to a MPEG-Dynamic Adaptive Streaming over HTTP(MPEG-DASH) system. MPEG-DASH is a technology for performing videostreaming using a HyperText Transfer Protocol (HTTP), and one offeatures thereof is that appropriate coded data among a plurality ofpieces of coded data having mutually-different resolutions and the like,which is prepared in advance, is selected and transmitted in units ofsegments. The MPEG-DASH 1551 performs generation of a stream that iscompliant with the standard, transmission control of the stream, and thelike and uses the MPEG-2 Video 1541 to HEVC/H.265 (Multi-view) 1545described above for coding and decoding image data.

The memory interface 1517 is an interface for the external memory 1312.The data supplied from the image processing engine 1514 or the codecengine 1516 is supplied to the external memory 1312 through the memoryinterface 1517. In addition, the data read from the external memory 1312is supplied to the video processor 1332 (the image processing engine1514 or the codec engine 1516) through the memory interface 1517.

The multiplexing/demultiplexing unit (MUX DMUX) 1518 performsmultiplexing and demultiplexing of various kinds of data relating to animage such as a bit stream of coded data, image data, and a videosignal. A method of the multiplexing/demultiplexing is arbitrary. Forexample, at the time of performing the multiplexing, themultiplexing/demultiplexing unit (MUX DMUX) 1518 may not only arrange aplurality of pieces of data into one but also add predetermined headerinformation or the like to the data. In addition, at the time ofperforming the demultiplexing, the multiplexing/demultiplexing unit (MUXDMUX) 1518 may not only divide one piece of data into multiple parts butalso add predetermined header information or the like to each divideddata part. In other words, the multiplexing/demultiplexing unit (MUXDMUX) 1518 can convert the format of data through themultiplexing/demultiplexing process. For example, themultiplexing/demultiplexing unit (MUX DMUX) 1518 can convert a bitstream into a transport stream that is a bit stream of the transmissionformat or data (file data) of the recording file format by multiplexingthe bitstream. It is apparent that an inverse conversion thereof can beperformed by the demultiplexing process.

The network interface 1519 is an interface, for example, dedicated forthe broadband modem 1333, the connectivity 1321 (all FIG. 39), or thelike. The video interface 1520 is an interface, for example, dedicatedfor the connectivity 1321, the camera 1322 (all FIG. 39), or the like.

Next, an example of the operation of such a video processor 1332 will bedescribed. For example, when a transport stream is received from anexternal network through the connectivity 1321, the broadband modem 1333(all FIG. 39), or the like, the transport stream is supplied to themultiplexing/demultiplexing unit (MUX DMUX) 1518 through the networkinterface 1519, is demultiplexed, and is decoded by the codec engine1516. For the image data acquired by the decoding process performed bythe codec engine 1516, predetermined image processing is performed, forexample, by the image processing engine 1514, and a predeterminedconversion is performed by the display engine 1513. Then, resultantimage data is supplied, for example, to the connectivity 1321 (FIG. 39)or the like through the display interface 1512, and an image thereof isdisplayed on a monitor. In addition, the image data, for example,acquired by the decoding process performed by the codec engine 1516 isreceded by the codec engine 1516, is multiplexed by themultiplexing/demultiplexing unit (MUX DMUX) 1518, is converted into filedata, is output, for example, to the connectivity 1321 (FIG. 39) or thelike through the video interface 1520, and is recorded on variousrecording media.

In addition, the file data of the coded data acquired by coding theimage data, which is read from a recording medium not illustrated, forexample, by the connectivity 1321 (FIG. 39) or the like is supplied tothe multiplexing/demultiplexing unit (MUX DMUX) 1518 through the videointerface 1520, is demultiplexed, and is decoded by the codec engine1516. For the image data acquired by the decoding process performed bythe codec engine 1516, predetermined image processing is performed bythe image processing engine 1514 and a predetermined conversion isperformed by the display engine 1513. Then, resultant image data issupplied, for example, to the connectivity 1321 (FIG. 39) or the likethrough tire display interface 1512, and an image thereof is displayedon the monitor. In addition, the image data, for example, acquired bythe decoding process performed by the codec engine 1516 is recoded bythe codec engine 1516, is multiplexed by the multiplexing/demultiplexingunit (MUX DMUX) 1518, is converted into a transport stream, is supplied,for example, to the connectivity 1321, the broadband modem 1333 (allFIG. 39), or the like through the network interface 1519, and istransmitted to another apparatus not illustrated.

Here, the transmission/reception of the image data and the other databetween each processing unit arranged inside the video processor 1332,for example, is performed using the internal memory 1515 or the externalmemory 1312. In addition, the power management module 1313, for example,controls the supply of power to the control unit 1511.

In a case where the present technology is applied to the video processor1332 configured as such, the present technology according to eachembodiment described above may be applied to the codec engine 1516. Inother words, for example, the codec engine 1516 may be configured toinclude the functional blocks realizing the coding apparatus and thedecoding apparatus according to the first and second embodiments. By thecodec engine 1516 configuring as such, the video processor 1332 canacquire advantages similar to the advantages described above withreference to FIGS. 1 to 20.

In addition, in the codec engine 1516, the present technology (in otherwords, the functions of the image coding apparatus and the imagedecoding apparatus according to each embodiment described above) may berealized by hardware such as logic circuits or software such as anembedded program or may be realised by both the hardware and thesoftware.

While two examples of the configuration of the video processor 1332 havebeen described as above, the configuration of the video processor 1332is arbitrary and may be a configuration other than the two examplesdescribed above. Here, the video processor 1332 may be configured aseither one semiconductor chip or a plurality of semiconductor chips. Forexample, the video processor may be configured as a three-dimensionalstacked LSI. In addition, the video processor may be realized by aplurality of LSIs.

(Example of Application to Apparatus)

The video set 1300 may be built in various apparatuses that processimage data. For example, the video set 1300 may be built in thetelevision apparatus 900 (FIG. 35), the mobile phone 920 (FIG. 36), therecording/reproducing apparatus 940 (FIG. 37), the imaging apparatus 960(FIG. 38), and the like. By building the video set 1300 therein, theapparatus can acquire advantages similar to those described above withreference to FIGS. 1 to 20.

Furthermore, some of the configurations of the video set 1300 describedabove may be configurations to which the present technology is appliedin a case where the video processor 1332 is included therein. Forexample, only the video processor 1332 may be configured as a videoprocessor to which the present technology is applied. In addition, asdescribed above, the processor, the video module 1311, and the likedenoted by the dotted line 1341 may be configured as a processor, amodule, and the like to which the present technology is applied.Furthermore, for example, the video module 1311, the external memory1312, the power management module 1313, and the front end module 1314may be combined so as to be configured as a video unit 1361 to which thepresent technology is applied. In any of the configurations, the sameadvantages as those described above with reference to FIGS. 1 to 20 canbe acquired.

In other words, any configuration that includes the video processor1332, similarly to the case of the video set 1300, may be built invarious apparatuses that process image data. For example, the videoprocessor 1332, the processor denoted by the dotted line 1341, the videomodule 1311, or the video unit 1361 may be built in the televisionapparatus 900 (FIG. 35), the mobile phone 920 (FIG. 36), therecording/reproducing apparatus 940 (FIG. 37), the imaging apparatus 960(FIG. 38), or the like. By building any configuration to which thepresent technology is applied into an apparatus, similarly to the caseof the video set 1300, the apparatus can acquire the same advantages asthose described above with reference to FIGS. 1 to 20.

In this specification, an example has been described in which variouskinds of information such as the identification data, is multiplexed inthe coded data and is transmitted from the coding side to the decodingside. However, a technique for transmitting such information is notlimited to such a technique. For example, such information may betransmitted or recorded as individual data associated with a coded datawithout being multiplexed in the coded data. Here, the term “associated”represents that an image (it may be a part of an image such as a slice,block, or the like) included in a bit stream and informationcorresponding to the image are acquired with being linked to each otherat the time of decoding the image and the information. In other words,the information may be transmitted in a transmission line other thanthat of the coded data. In addition, the information may be recorded ona recoding medium other than that for the coded data for a differentrecording area of the same recording medium). Furthermore, theinformation and the coded data, for example, may be associated with eachother in units of arbitrary parts such as multiple frames, one frame, ora part of the frame.

The present disclosure may be applied to a coding apparatus and adecoding apparatus that are used, as in MPEG, H.26x, or the like, when abit stream compressed through an orthogonal transform such as a discretecosine transform and motion compensation is received through a networkmedium such as satellite broadcasting, a cable TV, the internet, or themobile phone or when the compressed bit stream is processed on a storagemedium such as an optical disc, a magnetic disk, or a flash memory.

Furthermore, the coding system according to the present disclosure maybe a coding system other than the HEVC system that performs coding in aunit having a recursive hierarchical structure.

In this specification, a system represents a set of a plurality ofconstituent elements (an apparatus, a module (component), and the like),and all the constituent elements do not need to be disposed in a samecasing. Thus, a plurality of apparatuses that are housed in separatecasings and are connected through a network and one apparatus in which aplurality of modules are housed in one casing are systems.

In addition, advantages described in this specification are merelyexamples, and the advantages are not limited thereto, but there may beanother advantage.

An embodiment according to the present disclosure is not limited to theembodiment described above, but various changes may be made therein in arange not departing from the concept of the present disclosure.

For example, the present disclosure may take a configuration of cloudcomputing in which one function is divided and processed cooperativelyby a plurality of apparatuses through a network.

In addition, each step described in each flowchart described above maybe either executed by one apparatus or executed by a plurality ofapparatuses in a shared manner.

Furthermore, in a case where a plurality of processes are included inone step, the plurality of processes included in the one step may beeither executed by one apparatus or executed by a plurality ofapparatuses in a shared manner.

The present disclosure may also take the following configurations.

(1)

A decoding apparatus including

a decoding unit that decodes a bit stream coded according to a codingstandard having a profile in which a lowest compression rate at the timeof coding an image is set for each of a plurality of tiers in units ofblocks that are recursively divided.

(2)

The decoding apparatus described above in (1),

wherein the lowest compression rate set for each of the plurality oftiers is different for each level.

(3)

The decoding apparatus described above in (1) or (2),

wherein the plurality of tiers are a main tier and a high tier.

(4)

The decoding apparatus described above in (3),

wherein the lowest compression rate of the high tier of a level that isa predetermined level or a higher level and the lowest compression rateof the main tier are different from each other.

(5)

The decoding apparatus described above in (4),

wherein the predetermined level is level 5.

(6)

The decoding apparatus described above in (5),

wherein the lowest compression rate of the high tier of a level that islevel 5 or a higher level is “4”.

(7)

The decoding apparatus described above in one of (1) to (6),

wherein the coding standard is an H.265/HEVC standard, and the decodingunit decodes the coded bit stream according to the H.265/HEVC standard.

(8)

A decoding method using a decoding apparatus including

a decoding step of decoding a bit stream coded according to a codingstandard having a profile in which a lowest compression rate at the timeof coding an image is set for each of a plurality of tiers in units ofblocks that are recursively divided.

(9)

A coding apparatus including

a coding unit that codes an image according to a coding standard havinga profile in which a lowest compression rate at the time of coding animage is set for each of a plurality of tiers in units of blocks thatare recursively divided.

(10)

The coding apparatus described above in (9),

wherein the lowest compression rate set for each of the plurality oftiers is different for each level.

(11)

The coding apparatus described above in (9) or (10),

wherein the plurality of tiers are a main tier and a high tier.

(12)

The coding apparatus described above in (11),

wherein the lowest compression rate of the high tier of a level that isa predetermined level or a higher level and the lowest compression rateof the main tier are different from each other.

(13)

The coding apparatus described above in (12),

wherein the predetermined level is level 5.

(14)

The coding apparatus described above in (13),

wherein the lowest compression rate of the high tier of a level that islevel 5 or a higher level is “4”.

(15)

The coding apparatus described above in one of (9) to (14),

wherein the coding standard is an H.265/HEVC standard, and the codingunit codes the image according to the H.265/HEVC standard.

(16)

A coding method using a coding apparatus including

a coding step of coding an image according to a coding standard having aprofile in which a lowest compression rate at the time of coding animage is set for each of a plurality of tiers in units of blocks thatare recursively divided.

REFERENCE SIGNS LIST

-   10 Coding apparatus-   11 Setting unit-   12 Coding unit-   110 Decoding apparatus-   112 Extraction unit-   114 Decoding unit

1. A decoding apparatus comprising: a decoding unit that decodes a bitstream coded according to a coding standard having a profile in which alowest compression rate at the time of coding an image is set for eachof a plurality of tiers in units of blocks that are recursively divided.2. The decoding apparatus according to claim 1, wherein the lowestcompression rate set for each of the plurality of tiers is different foreach level.
 3. The decoding apparatus according to claim 2, wherein theplurality of tiers are a main tier and a high tier.
 4. The decodingapparatus according to claim 3, wherein the lowest compression rate ofthe high tier of a level that is a predetermined level or a higher leveland the lowest compression rate of the main tier are different from eachother.
 5. The decoding apparatus according to claim 4, wherein thepredetermined level is level
 5. 6. The decoding apparatus according toclaim 5, wherein the lowest compression rate of the high tier of a levelthat is level 5 or a higher level is “4”.
 7. The decoding apparatusaccording to claim 6, wherein the coding standard is an H.265/HEVCstandard, and wherein the decoding unit decodes the coded bit streamaccording to the H.265/HEVC standard.
 8. A decoding method using adecoding apparatus, the decoding method comprising: a decoding step ofdecoding a bit stream coded according to a coding standard having aprofile in which a lowest compression rate at the time of coding animage is set for each of a plurality of tiers in units of blocks thatare recursively divided.
 9. A coding apparatus comprising: a coding unitthat codes an image according to a coding standard having a profile inwhich a lowest compression rate at the time of coding an image is setfor each of a plurality of tiers in units of blocks that are recursivelydivided.
 10. The coding apparatus according to claim 9, wherein thelowest compression rate set for each of the plurality of tiers isdifferent for each level.
 11. The coding apparatus according to claim10, wherein the plurality of tiers are a main tier and a high tier. 12.The coding apparatus according to claim 11, wherein the lowestcompression rate of the high tier of a level that is a predeterminedlevel or a higher level and the lowest compression rate of the main tierare different from each other.
 13. The coding apparatus according toclaim 12, wherein the predetermined level is level
 5. 14. The codingapparatus according to claim 13, wherein the lowest compression rate ofthe high tier of a level that is level 5 or a higher level is “4”. 15.The coding apparatus according to claim 14, wherein the coding standardis an H.265/HEVC standard, and wherein the coding unit codes the imageaccording to the H.265/HEVC standard.
 16. A coding method using a codingapparatus, the coding method comprising: a coding step of coding animage according to a coding standard having a profile in which a lowestcompression rate at the time of coding an image is set for each of aplurality of tiers in units of blocks that are recursively divided.